From f3d337c2c04d1a22f28ff88f75ed111e7c464884 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 18:54:46 -0400 Subject: [PATCH 01/63] docs: add plan for the Map2Check C runtime cutover to the Rust library --- .../2026-10-15-cutover-runtime-c-map2check.md | 1429 +++++++++++++++++ 1 file changed, 1429 insertions(+) create mode 100644 docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md diff --git a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md new file mode 100644 index 0000000..80bcebb --- /dev/null +++ b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md @@ -0,0 +1,1429 @@ +# Cutover do runtime C do Map2Check para a biblioteca Rust — Plano de implementação + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Substituir o runtime C do Map2Check (`hbgit/Map2Check:modules/backend/library`) pela biblioteca Rust deste repositório (`libmap2check`) e remover o runtime C, sem regressão de veredito nem de desempenho e com rollback possível até um ponto de não-retorno explícito. + +**Architecture:** O runtime C é compilado em 23 arquivos `.bc` (LLVM 16), e o frontend (`modules/frontend/caller.cpp:226`) os liga ao programa instrumentado com `llvm-link`, escolhendo o conjunto por modo (memtrack, memcleanup, overflow, assert, none), por gerador (none, KLEE, LibFuzzer), por estrutura de dados (Array ou BTree) e por wasm. A biblioteca Rust passa a ser o único runtime. A biblioteca só exporta a sua API núcleo, sem camada de compatibilidade (D5). Primeiro ela ganha paridade funcional. Depois, numa branch do Map2Check, os passes LLVM passam a emitir diretamente a API núcleo e o frontend passa a ligar `libmap2check.a` (migração direta, sem ABI dupla). A branch é validada contra o C por testes diferenciais e mergeada num único merge commit, revertível. Só depois de um ciclo de release o C é removido. + +**Tech Stack:** Rust 1.82 (LLVM 19), cbindgen 0.26, serde_json; Map2Check: C++17, CMake, LLVM/Clang 16, KLEE (z3/stp), LibFuzzer, wabt 1.0.41, benchexec. + +**Spec:** a mensagem do usuário que pediu este plano (cutover para Rust apenas), mais as respostas da sessão de 2026-10-15: **escopo = runtime C do Map2Check**; **programas C de teste da ABI deste repositório ficam como testes**; **D5: sem camada de compatibilidade** e **migração direta dos passes, sem tabela de ABI dupla** (decisões do usuário). Requisitos de paridade de referência: [TODO.md](../../../TODO.md), issues #6–#24 e #36–#38, [docs/api-deltas.md](../../api-deltas.md) e [docs/semantic-deltas.md](../../semantic-deltas.md). + +## Global Constraints + +- Nada em C é removido sem equivalente Rust validado pelo gate diferencial (Task 4.2) no fluxo correspondente. +- Nenhuma funcionalidade é declarada obsoleta sem uma justificativa registrada em `docs/cutover/decisoes.md`, com evidência e aprovação explícita do mantenedor. +- Passos pequenos e reversíveis: até a Fase 3, `develop` do Map2Check não muda de runtime; da Task 5.1 até o ponto de não-retorno (Fase 6), voltar ao C é um `git revert -m 1` do merge. +- Regressão de desempenho máxima aceitável: **10% na mediana** do tempo de parede por tarefa (TODO.md, "Medir e controlar a regressão de desempenho"). +- Toolchain do Map2Check: **LLVM/Clang 16** (Dockerfile.dev:45-55, cmake/FindLibFuzzer.cmake:22). Toolchain deste repositório: **Rust 1.82.0 com LLVM 19.1.1** (`rust-toolchain.toml`). +- Commits com assinatura verificada; tags de preservação assinadas (`git tag -s`). +- Sem atribuição de IA em commits, PRs ou issues ([CLAUDE.md](../../../CLAUDE.md)). +- O header `include/map2check.h` é gerado por `build.rs`; nunca editá-lo à mão. +- **Sem camada de compatibilidade (D5):** a biblioteca Rust não exporta os nomes nem as assinaturas do runtime C. Nada de feature `compat`, módulo `compat.rs` ou aliases. Quem se adapta são os passes do Map2Check. +- **Migração direta:** os passes emitem só a API núcleo; não há tabela de ABI C nem opção de build para voltar ao C. O rollback até a Fase 6 é `git revert -m 1` do merge de `feat/rust-runtime` (Task 5.1). +- Branch de integração: `develop` nos dois repositórios (aqui, `main` é a branch de release). No Map2Check o padrão local é `master`; o PR vai contra `develop`. + +## Review Focus + +1. **Tarefa que gera `VERIFICATION SUCCEEDED`:** hoje, só o caminho KLEE (`--nondet-generator symex`) pode provar TRUE (`modules/frontend/map2check.cpp:270-285`). Se o runtime Rust não rodar sob KLEE, toda tarefa TRUE vira silêncio ou UNKNOWN. Esperado: a mesma contagem de TRUE que o C. O teste está na Task 4.2 (o gate exige paridade por gerador, incluindo `symex`). +2. **Duas execuções no mesmo diretório de trabalho:** o runtime C sobrescreve `map2check_property`, mas deixa arquivos antigos de propriedade (`map2check_property_free`, ...) que o frontend lê em ordem fixa (`tools.cpp:114-204`). Esperado: o veredito depende só da execução atual. O teste está na Task 2.4 (`stale_property_files_are_removed_on_init`). +3. **Iterações sucessivas do LibFuzzer no mesmo processo:** o estado de uma entrada contamina a seguinte. Esperado: cada `LLVMFuzzerTestOneInput` parte do estado zerado. O teste está na Task 2.1 (#9). +4. **Programa sem violação que chama `exit()` no meio, com `--memcleanup-property`:** o C verifica vazamento no `atexit`. Esperado: FALSE-MEMCLEANUP quando há bloco alcançável não liberado. O teste está na Task 2.2 (`memcleanup_reports_leak_at_exit`). +5. **Arquivo `.bc` Rust ligado com `llvm-link-16`:** bitcode do LLVM 19 não é lido pelo LLVM 16 ("Unknown attribute kind" / "Invalid record"). Esperado: o build falha alto e cedo, não na hora da verificação. O teste está na Task 1.1 (spike) e na Task 3.1 (o CMake falha se o artefato não for legível por `llvm-dis-16`). + +--- + +## 0. Achados que corrigem a premissa do pedido + +- **Neste repositório já não há C legado.** A implementação C/C++ foi removida em `f1cbcfe` (84 arquivos, 8 083 linhas), e a remoção já está em `develop` e em `main`. O último commit com o C é `f4c43b5`. A tag `archive/memory_check` aponta para um commit de 2021 e **não** preserva esse estado. O que resta em C são três programas consumidores da ABI (`test/code*_by_caller_test.c`) e o harness `tests/ffi_harness.c`. Por decisão do usuário, eles **ficam** como testes. +- **O runtime C ainda em uso está no Map2Check**, em `modules/backend/library` (3 893 linhas em 33 arquivos). Ele não é a mesma API desta biblioteca: os nomes divergem (`map2check_binop_add` × `map2check_binop_add_int`, `map2check_assert` × `map2check_is_valid_assert`, `map2check_malloc` × `map2check_map_malloc`, `map2check_non_det_int` × `__VERIFIER_nondet_int`). Dos nomes `map2check_*` que aparecem nos passes, cerca de 30 não existem no header Rust. A lista `audit/evidence/missing_compat.txt` dá 33, mas está inflada: `map2check_pointer_cast` (`OverflowPass.cpp:255`) e `map2check_nondet_cast_` (`NonDetPass.cpp:167`) são só nomes de valores do IR (`Twine` de instruções de cast), não chamadas ao runtime. Recalcular na Task 0.3, porque a evidência é de `Map2Check@f01e8c3` e o `master` local está em `83471fba`. Como não haverá camada de compatibilidade (D5), essa diferença se resolve mudando o que os passes emitem (Task 3.2), não a biblioteca. +- **O runtime é ligado como bitcode, não como `.a`.** `caller.cpp:226-312` faz `llvm-link` de `.bc` no programa instrumentado, e esse `.bc` final vai para `clang -fsanitize=fuzzer` ou para o KLEE. O rustc 1.82 emite bitcode LLVM 19, que o `llvm-link-16` não lê. **Esse é o bloqueio técnico central** (Fase 1). +- **O runtime C é composto por modo na hora do link.** Cada `AnalysisMode*.c` define os mesmos símbolos (`analysis_init`, `map2check_malloc`, ...), e o frontend escolhe qual `.bc` ligar. Uma biblioteca Rust única precisa selecionar o modo em tempo de execução. +- **O canal de resultado é por arquivos, não JSON.** O frontend lê `map2check_property`, `map2check_property_{free,deref,assert,overflow,memtrack,memcleanup,unknown}`, `list_log.csv`, `klee_log.csv`, `track_bb_log.st` e `map2check_checked_error` (`tools.hpp:43-54`, `tools.cpp:114-204`, `caller.cpp:44`). +- **Rust → C via FFI:** o único `extern "C"` importado é `__map2check_main__` (`rust/src/nondet/libfuzzer.rs:44`), o `main` renomeado do programa analisado. Não é código C legado, então **permanece por design**. Se o KLEE for adotado, entram também `klee_make_symbolic` e `klee_assume`, que são intrínsecos do KLEE e também permanecem. A libc chega via `std` (malloc, exit), e isso é aceitável. Não há C próprio chamado pelo Rust a eliminar. + +## 1. Mapa de paridade + +Status: **(a)** portado · **(b)** parcial · **(c)** não portado · **(d)** candidato a obsoleto (sujeito à decisão registrada). Esforço: **P** ≤ 1 dia · **M** 2–4 dias · **G** 1–2 semanas · **GG** > 2 semanas. Responsável: o repositório não registra equipe além do mantenedor (Herbert Rocha); a coluna "Ordem" dá a sequência e o mantenedor atribui pessoas ao aprovar o plano. + +| # | Módulo C (Map2Check `modules/backend/library/lib/`) | Equivalente Rust | Status | Esforço | Depende de | Ordem (Task) | +|---|---|---|---|---|---|---| +| 1 | `Map2CheckFunctions.c`: init(isSvComp), passo, assume, `crab_assume`, `assume_loop`, `track_bb`, `__VERIFIER_error`, `target_function`, success/error/destroy/exit | `ffi.rs` (init, success, passo, assume, `__VERIFIER_error`) | b | M | 2.3 (modo), 3.2 (passes) | 2.3, 2.5, 3.2 | +| 2 | `AnalysisModeAssert.c` | `analysismode/assert.rs` | b (nomes/assinatura: muda o passe) | P | 3.2 | 3.2 | +| 3 | `AnalysisModeOverflow.c` (add/sub/mul/sdiv int e uint) | `analysismode/overflow.rs` | b (nomes: muda o passe; `sdiv` → `div_int`) | P–M | 3.2 | 3.2 | +| 4 | `AnalysisModeMemtrack.c` + `AllocationLog.c` + `ListLog.c` + `HeapLog.c` | `memtrack/`, `analysismode/memory.rs` | b (bugs #11, #36, #37, #38; CSVs) | G | 2.1 | 2.1, 2.4 | +| 5 | `AnalysisModeMemcleanup.c` | stub `map2check_set_memcleanup` | c (#10) | M | 2.1 | 2.2 | +| 6 | `AnalysisModeNone.c` | — | c (trivial) | P | 2.3 | 2.3 | +| 7 | `ContainerRealloc.c` | `Vec` em `AnalysisState` | a (por design) | P (verificar) | — | 4.2 | +| 8 | `ContainerBTree.c` + `BTree.c` (persistência em `*.map2check.bin`; opção `--btree`, marcada "experimental") | — | d? (decisão D3) | P se justificado; M se portado | 4.3 | 2.6 | +| 9 | `NonDetLog.c` → `klee_log.csv` | `nondet/mod.rs` (JSON) | b (formato) | M | 2.4 | 2.4 | +| 10 | `NonDetGeneratorNone.c` | `__VERIFIER_nondet_*` sem feature | b | P | 2.5, 3.2 | 2.5 | +| 11 | `NonDetGeneratorLibFuzzy.c` | `nondet/libfuzzer.rs` | b (#12, #6) | M | 2.1 | 2.5 | +| 12 | `NonDetGeneratorKlee.c` (`klee_make_symbolic`, `klee_assume`) | — (feature `klee` removida do `Cargo.toml`) | c (#20) | G–GG | **1.1 (bloqueante)** | 2.7 | +| 13 | `PropertyGenerator.c` → `map2check_property*` | `output/` (só JSON) | c (#19) | M | 2.3 | 2.4 | +| 14 | `TrackBBLog.c` → `track_bb_log.st` | `bbtrack/` | b (formato) | M | 2.4 | 2.4 | +| 15 | `WitnessGeneration.c` / `WitnessGenerationNone.c` → `map2check_checked_error` | — | c | M | 2.4 | 2.4 | +| 16 | `WasmRuntimeStubs.c` (`wasm_rt_*`) + `map2check_wasm_{malloc,free,check_access}` | — | c (decisão D2) | M | 1.1, 2.5 | 2.8 | +| 17 | `header/*.h` (13 headers) | `include/map2check.h` gerado | a (gerado) | P | 2.5 | 6.2 | +| 18 | `tests/unit/backend-library/*.cpp` (gtest: AllocationLog, BTree, Container, HeapLog, MemTrack) | `ffi_tests.rs` + testes de módulo | b (portar asserções) | P–M | 2.1, 2.6 | 5.2 | +| 20 | Passes LLVM: nomes e assinaturas das chamadas de runtime (`modules/backend/pass/*.cpp`) | — (C++, permanece) | c (emitem a ABI C) | G | 2.5 | 3.2 | +| 19 | `map2check_posix` (marcado `__attribute_deprecated__`, `AnalysisModeMemtrack.c:327`, mas ainda chamado por `MemoryTrackPass.cpp:120` para `posix_memalign`) | — (vira `map2check_map_posix_memalign`) | c (não é obsoleto: há chamador) | P | 2.5 | 2.5 | + +Itens fora deste repositório que **continuam em C++** por decisão de escopo: os passes LLVM (`modules/backend/pass`) e o frontend (`modules/frontend`). Eles mudam no necessário para ligar o runtime Rust e para emitir a API núcleo (Tasks 3.2 e 3.3). + +## 2. Decisões (as pendentes bloqueiam as tarefas indicadas) + +| ID | Decisão | Opções | Recomendação | Bloqueia | +|---|---|---|---|---| +| D1 | Como o runtime Rust entra no pipeline LLVM 16 | (A) emitir bitcode legível pelo LLVM 16; (B) ligar `libmap2check.a` nativo no passo final do `clang`, sem `llvm-link`; (C) migrar o Map2Check para LLVM 19 | **B** para None/LibFuzzer; para KLEE, só o spike decide (A ou C). B não depende da versão do LLVM, porque o objeto é nativo. | 1.2, 2.7, 3.x | +| D2 | O modo `--wasm` entra no cutover? | portar os stubs para Rust; ou manter fora do escopo | Portar. O CI do Map2Check tem o job `e2e-wasm` ativo (`ci.yml:213`), então não há base para declará-lo obsoleto. | 2.8 | +| D3 | A opção `--btree` (disco) | portar; ou remover com justificativa | Medir antes (Task 4.3). Se o RSS do Rust nos casos grandes ficar ≤ o RSS do C com `--btree`, remover e registrar a justificativa. | 2.6 | +| D4 | Limite de memória (pico de RSS) | — | ≤ 1,25× o C na mediana. O TODO só fixa o limite de tempo; este valor precisa de aprovação. | 4.2 | +| D5 | A ABI final que os passes emitem | **Decidida em 2026-10-15: sem camada de compatibilidade.** Os passes migram diretamente para a API núcleo, sem ABI dupla (#23 entra no cutover); a #21 fecha como "won't do". | — | 0.3, 2.5, 3.2 | + +## 3. Critérios de aceitação do cutover + +Os critérios são medidos pelo gate da Task 4.2 sobre o corpus `tests/regression_test/test_cases/sv-benchmarks` (39 tarefas `.yml`), em cada combinação **propriedade × gerador** (`fuzzer`, `symex`), com 5 repetições, na mesma imagem `Dockerfile.dev` e na mesma máquina. + +**Funcionais (qualquer violação bloqueia):** +1. O veredito (`SUCCEEDED`/`FAILED`/`UNKNOWN`) é igual ao do C em todas as tarefas, por moda das 5 repetições. +2. Para FAILED, a primeira linha de `map2check_property` (tag), `Line:` e `Function:` são iguais. +3. Onde o C gerou `map2check_checked_error` ou witness, o Rust também gera. +4. A contagem de tarefas com `expected_verdict` correto não diminui. +5. Qualquer divergência só passa se estiver listada em `docs/cutover/deltas-aceitos.tsv` com a referência de um item `SD-xx` em `docs/semantic-deltas.md` que a classifique como "bug do C corrigido", com caso reproduzível. + +**Desempenho:** +6. A mediana, entre tarefas, da razão `wall_rust / wall_c` é ≤ 1,10. **Bloqueia.** +7. A contagem de TIMEOUT por gerador não aumenta. **Bloqueia.** +8. A mediana da razão de pico de RSS é ≤ 1,25 (D4). **Bloqueia** depois de aprovado; antes disso, só alerta. +9. Uma tarefa isolada com razão > 1,5 não bloqueia se os critérios 6 e 7 passarem, mas exige perfil registrado em `docs/cutover/perf.md`. + +**Qualidade da biblioteca (já existentes; continuam obrigatórios):** `cargo test --all-features`, `cargo clippy -- -D warnings`, `cargo fmt --check`, `cargo deny check`, gate de cobertura e `tests/ffi_harness.c` sob ASan/UBSan sem achados. + +**Prontidão (#24):** os critérios 1–8 passam em **duas** execuções completas consecutivas do gate, separadas por pelo menos um merge em `develop` do Map2Check, antes do merge da Task 5.1 e de novo antes da Fase 6. + +## 4. Arquivos e diretórios + +### A remover (só na Fase 6, no Map2Check) + +| Caminho | Pré-condição | +|---|---| +| `modules/backend/library/lib/*.c`, `lib/BTree.h`, `lib/NOTE.txt`, `lib/CMakeLists.txt` | Task 5.1 concluída (um ciclo de release sem revert); o código já está fora do build desde a Task 3.1 | +| `modules/backend/library/header/*.h` | nenhum `#include` restante (`grep -r 'backend/library/header'`) | +| `modules/backend/library/CMakeLists.txt`, `NOTES.txt` | idem | +| `tests/unit/backend-library/` | asserções portadas (Task 5.2); já fora do `ctest` desde a Task 3.4 | +| referências restantes ao runtime C em configs de análise (`.cppcheck-suppressions.txt`, `.clang-tidy`) | Task 6.3 | + +### A manter + +| Caminho | Motivo | +|---|---| +| Neste repo: `rust/`, `tests/*.rs`, `build.rs`, `cbindgen.toml`, `include/map2check.h` | a implementação | +| Neste repo: `test/code{API,fuzz,klee}_by_caller_test.c`, `tests/ffi_harness.c` | testes de ABI do ponto de vista de um compilador C (decisão do usuário) | +| Neste repo: `audit/`, `docs/plano_map2check_library_rust.md` | histórico/evidência; atualizar links quebrados (Task 6.4) | +| Map2Check: `modules/backend/pass/`, `modules/frontend/` | fora do escopo (C++, não runtime) | +| Map2Check: `tests/regression_test/`, `tests/integration/` | corpus do gate diferencial | + +## 5. Riscos e mitigações + +| Risco | Evidência | Mitigação | Task | +|---|---|---|---| +| Bitcode LLVM 19 ilegível pelo `llvm-link-16` | `rustc -vV` → LLVM 19.1.1; Map2Check usa LLVM 16 | spike com 3 rotas e critério objetivo; o CMake verifica com `llvm-dis-16` | 1.1, 3.1 | +| KLEE não executa um runtime com `std` (TLS, `Mutex`, unwinding, alocador) | o runtime C atual é C puro, sem libc pesada | o spike mede uma variante `no_std + alloc` com panic=abort; sem ela, sem cutover do `symex` | 1.1, 2.7 | +| Sem KLEE, nenhuma tarefa TRUE é provada | `map2check.cpp:270-285` | o KLEE é condição de cutover, não opcional (Review Focus 1) | 4.2 | +| Typo implícito: o C grava `map2check_property__unknown` (dois `_`), mas o frontend lê `map2check_property_unknown` | `PropertyGenerator.c:16`, `tools.cpp:114` | reproduzir o comportamento observável (o frontend nunca vê UNKNOWN por esse arquivo) e registrar como SD; não "consertar" no cutover | 2.4 | +| Arquivos de propriedade antigos no diretório de trabalho mudam o veredito | o C só sobrescreve os que escreve | limpar no `init`; o caller usa um diretório por hash, mas o teste garante isso | 2.4 | +| O estado do C é por arquivo `.bc` (globais estáticos por modo); o Rust tem um estado único | `caller.cpp:254-312` | `map2check_set_mode()` antes de `map2check_init`; teste de exclusão mútua | 2.3 | +| `isSvComp` de `map2check_init(int)` mudaria o comportamento | `Map2CheckFunctions.c:29-36`: `sv_comp` é gravado e nunca lido em `library/lib` | argumento morto; o passe deixa de passá-lo; justificativa registrada em `abi-mapping.tsv` e em `docs/cutover/decisoes.md` | 0.3, 3.2 | +| A ordem de avaliação no `atexit` (memcleanup) difere | `AnalysisModeMemcleanup.c` | teste com `exit()` no meio do programa | 2.2 | +| Assinaturas divergentes entre as ABIs (`map2check_load(ptr, i64)` × `map2check_check_load(ptr, line, scope, size, fn)`, `map2check_nondet_` × `map2check_save_nondet_log_`, `map2check_check_deref`) | `MemoryTrackPass.cpp:739`, `NonDetFunctions.hpp:46`, TODO.md | o passe emite a assinatura núcleo; `verifyModule` + teste de IR por hook + teste de link; argumento errado no IR não dá erro de link, só comportamento errado, por isso o gate 4.2 é obrigatório | 3.2, 4.2 | +| Sem ABI dupla, não há rollback por flag: depois do merge, voltar ao C exige revert | migração direta | trabalho isolado em `feat/rust-runtime`; merge único com merge commit; gate 4.2 duas vezes antes do merge; revert ensaiado no PR (Task 5.1); a branch é rebaseada semanalmente para que o revert continue limpo | 3.x, 5.1 | +| Um passe chama função inexistente na biblioteca: o `clang` final só falha com `undefined reference` na hora da verificação | — | gate `symbol-matrix.sh` + `core_api_coverage` + teste de link ponta a ponta no CI do Map2Check | 0.3, 2.5, 3.2 | +| Símbolos duplicados se C e Rust forem ligados juntos | os dois definem `__VERIFIER_*` | o build deixa de compilar/instalar os `.bc` C; o teste da Task 3.1 falha se algum for instalado | 3.1 | +| Dependências externas: wabt 1.0.41 em `/opt`, klee-uclibc compilado com clang-16, solvers z3/stp/btor/yices2 | `Dockerfile.dev`, `caller.cpp:389-420` | gate em container `Dockerfile.dev` + toolchain Rust instalada nele; versões fixadas | 3.3 | +| Diferença de alocador/endereços muda contraexemplos textuais | endereços aparecem na saída | o gate compara campos normalizados, não o texto | 4.1 | +| Unwinding atravessando a FFI | #8 | `panic = "abort"` no perfil de runtime | 2.1 | + +--- + +## Fase 0 — Preservação e linha de base (totalmente reversível) + +### Task 0.1: Tags de preservação do legado + +**Files:** nenhum arquivo; cria tags nos dois repositórios. + +**Interfaces:** +- Produces: tags `legacy/c-library-final` (este repo) e `legacy/c-runtime-final` (Map2Check), mais a branch `legacy/c-runtime` (Map2Check), que são os pontos de rollback citados nas Fases 5 e 6. + +- [ ] **Step 1: Confirmar o último commit com o C neste repositório** + +Run: `git -C ~/Documents/Dev/map2check-library ls-tree -r --name-only f1cbcfe^ | grep -cE '^(src|include|lib|old_src)/'` +Expected: um número > 0. `f1cbcfe^` é `f4c43b5`. + +- [ ] **Step 2: Criar e publicar a tag assinada aqui** + +```bash +cd ~/Documents/Dev/map2check-library +git tag -s legacy/c-library-final f1cbcfe^ -m "Última revisão com a implementação C da map2check-library (antes de f1cbcfe)" +git push origin legacy/c-library-final +``` + +- [ ] **Step 3: Criar a tag e a branch no Map2Check, no topo de `develop`** + +```bash +cd ~/Documents/Dev/Map2Check +git fetch origin +git tag -s legacy/c-runtime-final origin/develop -m "Última revisão com o runtime C em modules/backend/library" +git branch legacy/c-runtime origin/develop +git push origin legacy/c-runtime-final legacy/c-runtime +``` + +- [ ] **Step 4: Verificar** + +Run: `git -C ~/Documents/Dev/Map2Check ls-remote --tags origin legacy/c-runtime-final && git -C ~/Documents/Dev/map2check-library tag -v legacy/c-library-final` +Expected: a tag aparece no remoto e `tag -v` mostra "Good signature". + +**Critério de conclusão:** as duas tags assinadas e a branch existem no remoto. A tag do Map2Check será movida (`-f`) na Task 6.1 para o último commit antes da remoção, e esse movimento fica registrado no PR. + +### Task 0.2: Runner de corpus e linha de base do C + +**Files:** +- Create (Map2Check): `scripts/cutover/run-corpus.sh` +- Create (Map2Check): `scripts/cutover/test-run-corpus.sh` +- Create (Map2Check): `tests/cutover/baseline-c.tsv` (gerado) + +**Interfaces:** +- Produces: `run-corpus.sh `, com as variáveis `GENS` (padrão `"fuzzer symex"`), `REPS` (padrão `5`) e `TIMEOUT` (padrão `45`). Gera um TSV com o cabeçalho exato `task prp gen rep verdict tag line function witness wall_s maxrss_kb`, consumido pela Task 4.1. + +- [ ] **Step 1: Escrever o teste do runner com um `map2check` falso** + +```bash +#!/usr/bin/env bash +# Testa run-corpus.sh com um binário falso e um corpus de uma tarefa. +set -euo pipefail +HERE="$(cd "$(dirname "$0")" && pwd)" +T="$(mktemp -d)"; trap 'rm -rf "$T"' EXIT +mkdir -p "$T/bin" "$T/cases/x" +cat > "$T/bin/map2check" <<'EOF' +#!/usr/bin/env bash +printf 'FALSE-DEREF\nLine: 7\nFunction: main\n' > map2check_property +echo "VERIFICATION FAILED" +EOF +chmod +x "$T/bin/map2check" +printf 'int main(){}\n' > "$T/cases/x/a.c" +cat > "$T/cases/x/a.yml" <<'EOF' +format_version: '1.0' +input_files: 'a.c' +properties: + - property_file: ../properties/valid-memsafety.prp + expected_verdict: false +EOF +CASES="$T/cases" GENS=fuzzer REPS=1 "$HERE/run-corpus.sh" "$T/bin" "$T/out.tsv" +row="$(sed -n 2p "$T/out.tsv")" +IFS=$'\t' read -r task prp gen rep verdict tag line func witness wall rss <<<"$row" +[ "$task" = "x/a.yml" ] && [ "$prp" = "valid-memsafety" ] && [ "$gen" = fuzzer ] \ + && [ "$verdict" = FAILED ] && [ "$tag" = FALSE-DEREF ] && [ "$line" = 7 ] \ + && [ "$func" = main ] && [ "$witness" = 0 ] || { echo "linha inesperada: $row" >&2; exit 1; } +echo ok +``` + +- [ ] **Step 2: Rodar e ver falhar** + +Run: `bash scripts/cutover/test-run-corpus.sh` +Expected: FAIL com "No such file or directory" (o runner ainda não existe). + +- [ ] **Step 3: Implementar o runner** + +```bash +#!/usr/bin/env bash +# Executa o corpus de regressão e grava, por tarefa/propriedade/gerador/repetição, +# o veredito, a tag de map2check_property, linha, função, witness, tempo e RSS. +# Uso: run-corpus.sh +set -euo pipefail +BIN_DIR="$(cd "$1" && pwd)"; OUT="$2" +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +CASES="${CASES:-$ROOT/tests/regression_test/test_cases/sv-benchmarks}" +GENS="${GENS:-fuzzer symex}"; REPS="${REPS:-5}"; TIMEOUT="${TIMEOUT:-45}" + +flags_for() { + case "$1" in + valid-memsafety) echo "--memtrack" ;; + valid-memcleanup) echo "--memtrack --memcleanup-property" ;; + no-overflow) echo "--check-overflow" ;; + unreach-call) echo "--target-function" ;; + *) return 1 ;; + esac +} + +printf 'task\tprp\tgen\trep\tverdict\ttag\tline\tfunction\twitness\twall_s\tmaxrss_kb\n' > "$OUT" +while IFS= read -r yml; do + dir="$(dirname "$yml")" + input="$dir/$(sed -nE "s/^input_files: *'?([^' ]+)'?.*/\1/p" "$yml")" + while IFS= read -r prp; do + flags="$(flags_for "$prp")" || continue + for gen in $GENS; do + for rep in $(seq 1 "$REPS"); do + work="$(mktemp -d)" + # shellcheck disable=SC2086 + ( cd "$work" && /usr/bin/time -f '%e\t%M' -o time.txt \ + timeout "$TIMEOUT" "$BIN_DIR/map2check" --nondet-generator "$gen" $flags "$input" \ + > out.txt 2>&1 ) || true + verdict=TIMEOUT + grep -q 'VERIFICATION SUCCEEDED' "$work/out.txt" && verdict=SUCCEEDED + grep -q 'VERIFICATION FAILED' "$work/out.txt" && verdict=FAILED + grep -q 'VERIFICATION UNKNOWN' "$work/out.txt" && verdict=UNKNOWN + tag=-; line=-; func=- + if [ -f "$work/map2check_property" ]; then + tag="$(sed -n 1p "$work/map2check_property")" + line="$(sed -nE 's/^Line: *//p' "$work/map2check_property" | head -1)"; line="${line:--}" + func="$(sed -nE 's/^Function: *//p' "$work/map2check_property" | head -1)"; func="${func:--}" + fi + witness=0; ls "$work"/*.graphml "$work"/map2check_checked_error >/dev/null 2>&1 && witness=1 + timing="$(tail -1 "$work/time.txt" 2>/dev/null || printf -- '-\t-')" + printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ + "${yml#"$CASES"/}" "$prp" "$gen" "$rep" "$verdict" "$tag" "$line" "$func" "$witness" "$timing" >> "$OUT" + rm -rf "$work" + done + done + done < <(sed -nE 's#.*property_file: *.*/([a-z-]+)\.prp.*#\1#p' "$yml") +done < <(find "$CASES" -name '*.yml' | sort) +``` + +- [ ] **Step 4: Rodar o teste** + +Run: `bash scripts/cutover/test-run-corpus.sh` +Expected: `ok` + +- [ ] **Step 5: Gerar a linha de base do C na imagem `Dockerfile.dev`** + +```bash +docker build -f Dockerfile.dev -t map2check-dev . +docker run --rm -v "$PWD":/src -w /src map2check-dev \ + bash -c 'cmake -S . -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j && cmake --install build --prefix release && scripts/cutover/run-corpus.sh release/bin tests/cutover/baseline-c.tsv' +``` + +Expected: `tests/cutover/baseline-c.tsv` com cabeçalho + (tarefas×propriedades suportadas×2×5) linhas. Anotar em `tests/cutover/README.md` o hash do commit do Map2Check, o `docker image inspect --format '{{.Id}}'` e a CPU da máquina. + +- [ ] **Step 6: Commit (Map2Check)** + +```bash +git add scripts/cutover tests/cutover +git commit -S -m "test: add regression corpus runner and C runtime baseline for Rust cutover" +``` + +**Critério de conclusão:** o teste do runner passa e a linha de base foi gerada e versionada, com zero linhas `TIMEOUT` sem explicação. Se houver timeouts, registrá-los no README como estado conhecido do C. + +### Task 0.3: Mapeamento ABI C → API núcleo e matriz de símbolos + +**Files:** +- Create (este repo): `docs/cutover/abi-mapping.tsv` +- Create (este repo): `scripts/symbol-matrix.sh` +- Create (este repo): `scripts/test-symbol-matrix.sh` +- Create (este repo): `docs/cutover/symbol-matrix.tsv` (gerado) + +**Interfaces:** +- Consumes: os fontes dos passes do Map2Check (`modules/backend/pass/*.{cpp,hpp}`) e o header `include/map2check.h`. +- Produces: + - `docs/cutover/abi-mapping.tsv`, com o cabeçalho exato `c_symbol core_symbol owner_task notes`. É a especificação que a Task 2.5 (biblioteca) e a Task 3.2 (passes) implementam. `core_symbol` pode listar várias chamadas separadas por `+`, ou `-` quando o símbolo deixa de ser chamado (com a justificativa em `notes`). + - `symbol-matrix.sh `, que imprime o TSV `symbol core_symbol in_rust_header`. Sai com status 1 se algum símbolo emitido pelos passes não tiver linha no mapeamento ou se algum `core_symbol` não estiver no header. As Tasks 2.5 e 3.2 usam esse status como gate. + +- [ ] **Step 1: Escrever o mapeamento inicial** + +Ponto de partida, levantado em 2026-10-15 contra `Map2Check@83471fba`. As assinaturas C vêm das chamadas `getOrInsertFunction` nos passes; as Rust, de `rust/src/ffi.rs`. As linhas com `owner_task` ≠ `-` exigem export novo na biblioteca. + +```tsv +c_symbol core_symbol owner_task notes +map2check_init map2check_set_mode+map2check_init 2.3 o arg int isSvComp é morto: Map2CheckFunctions.c:35 grava sv_comp e nenhum arquivo o lê +map2check_success map2check_success - +map2check_track_bb map2check_save_basic_block_log - conferir a ordem dos args +map2check_target_function map2check_reach_target 2.3 +map2check_assert map2check_is_valid_assert - +map2check_assume __VERIFIER_assume - +map2check_assume_loop __VERIFIER_assume - o C faz cast de char para int; o passe emite zext +map2check_crab_assume __VERIFIER_assume - idem +map2check_binop_add map2check_binop_add_int - +map2check_binop_add_uint map2check_binop_add_unsigned - +map2check_binop_sub map2check_binop_sub_int - +map2check_binop_sub_uint map2check_binop_sub_unsigned - +map2check_binop_mul map2check_binop_mul_int - +map2check_binop_mul_uint map2check_binop_mul_unsigned - +map2check_binop_sdiv map2check_binop_div_int - +map2check_malloc map2check_map_malloc - +map2check_calloc map2check_map_calloc - +map2check_alloca map2check_map_alloca - +map2check_non_static_alloca map2check_map_non_static_alloca - SD-12 (#38): unidade de tamanho +map2check_function map2check_map_funct_address - +map2check_add_store_pointer map2check_map_store_pointer - +map2check_load map2check_check_load - o C recebe (ptr, i64 size); o núcleo recebe linha, escopo e função: o passe passa DebugInfo +map2check_free map2check_check_free+map2check_map_free - o C desreferencia ptr (AnalysisModeMemtrack.c:341); manter o mesmo nível de indireção +map2check_free_resolved_address map2check_check_free_resolved_address - SD-06 (#37) +map2check_posix map2check_map_posix_memalign 2.5 chamado em MemoryTrackPass.cpp:120 para posix_memalign; não é obsoleto +map2check_wasm_malloc map2check_wasm_malloc 2.8 +map2check_wasm_free map2check_wasm_free 2.8 +map2check_wasm_check_access map2check_wasm_check_access 2.8 +__map2check_main__ __map2check_main__ - definido pelo programa instrumentado, não pela biblioteca +__VERIFIER_assert __VERIFIER_assert - +__VERIFIER_assume __VERIFIER_assume - +__VERIFIER_error __VERIFIER_error - +``` + +A expansão por tipo de `map2check_non_det_` (→ `__VERIFIER_nondet_`) e de `map2check_nondet_` (→ `map2check_save_nondet_log_`) deve ser acrescentada, uma linha por tipo, a partir das macros `INSTRUMENT_HEADER_HELPER`/`CALL_IF_HELPER` de `modules/backend/pass/NonDetPass.hpp`. Os tipos sem equivalente núcleo hoje (`loff_t`, `sector_t`, `pchar`, `unsigned`, `short`, `uchar`, `size_t` no log) recebem `owner_task` = `2.5` (#6). `map2check_pointer_cast` e `map2check_nondet_cast_` **não** entram: são nomes de valores do IR (`OverflowPass.cpp:255`, `NonDetPass.cpp:167`), não chamadas. + +- [ ] **Step 2: Teste do script com passes falsos** + +```bash +#!/usr/bin/env bash +set -euo pipefail +HERE="$(cd "$(dirname "$0")" && pwd)" +T="$(mktemp -d)"; trap 'rm -rf "$T"' EXIT +mkdir -p "$T/modules/backend/pass" +cat > "$T/modules/backend/pass/P.cpp" <<'CPP' +f("map2check_init"); f("map2check_alloca"); Twine b("map2check_pointer_cast"); f("map2check_unmapped"); +CPP +printf 'c_symbol\tcore_symbol\towner_task\tnotes\nmap2check_init\tmap2check_init\t-\t\nmap2check_alloca\tmap2check_map_alloca\t-\t\n' > "$T/map.tsv" +out="$("$HERE/symbol-matrix.sh" "$T" "$T/map.tsv" || true)" +grep -qP '^map2check_init\tmap2check_init\t1$' <<<"$out" +grep -qP '^map2check_alloca\tmap2check_map_alloca\t1$' <<<"$out" +grep -qP '^map2check_unmapped\t-\t0$' <<<"$out" +! grep -q pointer_cast <<<"$out" +if "$HERE/symbol-matrix.sh" "$T" "$T/map.tsv" >/dev/null; then echo "deveria falhar" >&2; exit 1; fi +echo ok +``` + +- [ ] **Step 3: Rodar e ver falhar** + +Run: `bash scripts/test-symbol-matrix.sh` +Expected: FAIL (o script não existe). + +- [ ] **Step 4: Implementar** + +```bash +#!/usr/bin/env bash +# SPDX-License-Identifier: GPL-3.0-only +# Cruza os símbolos de runtime emitidos pelos passes do Map2Check, traduzidos +# por docs/cutover/abi-mapping.tsv, com os declarados em include/map2check.h. +# Sai com 1 se algum símbolo não tiver mapeamento ou se o alvo não existir. +# Uso: symbol-matrix.sh +set -euo pipefail +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +M2C="$1"; MAP="$2" +# Nomes de valores do IR (Twine de casts), não chamadas ao runtime. +IR_VALUE_NAMES='^(map2check_pointer_cast|map2check_nondet_cast_.*)$' +pass="$(grep -ohE '"(map2check_|__VERIFIER_|__map2check)[A-Za-z0-9_]*[A-Za-z0-9]"' \ + "$M2C"/modules/backend/pass/*.cpp "$M2C"/modules/backend/pass/*.hpp 2>/dev/null \ + | tr -d '"' | grep -vE "$IR_VALUE_NAMES" | sort -u)" +hdr="$(grep -oE '\b(map2check_[a-z0-9_]+|__VERIFIER_[a-z0-9_]+)\s*\(' \ + "$ROOT/include/map2check.h" | tr -d '( ' | sort -u)" +missing=0 +printf 'symbol\tcore_symbol\tin_rust_header\n' +while IFS= read -r s; do + [ -z "$s" ] && continue + core="$(awk -F'\t' -v s="$s" 'NR>1 && $1==s {print $2}' "$MAP")" + if [ -z "$core" ]; then printf '%s\t-\t0\n' "$s"; missing=1; continue; fi + ok=1 + if [ "$core" != "-" ] && [ "$core" != "__map2check_main__" ]; then + IFS='+' read -ra parts <<<"$core" + for c in "${parts[@]}"; do grep -qxF "$c" <<<"$hdr" || ok=0; done + fi + [ "$ok" = 1 ] || missing=1 + printf '%s\t%s\t%s\n' "$s" "$core" "$ok" +done <<<"$pass" +exit "$missing" +``` + +`__map2check_main__` é definido pelo programa instrumentado, então não é procurado no header. + +- [ ] **Step 5: Rodar o teste e gerar a matriz real** + +Run: `bash scripts/test-symbol-matrix.sh && (scripts/symbol-matrix.sh ~/Documents/Dev/Map2Check docs/cutover/abi-mapping.tsv > docs/cutover/symbol-matrix.tsv || true) && awk -F'\t' '$3==0' docs/cutover/symbol-matrix.tsv` +Expected: `ok`, e só aparecem como `0` as linhas cujo `owner_task` ainda não foi feito (2.3, 2.5, 2.8). Se aparecer um símbolo com `core_symbol` = `-`, falta uma linha no mapeamento: completar o Step 1. + +- [ ] **Step 6: Commit** + +```bash +git add docs/cutover/abi-mapping.tsv scripts/symbol-matrix.sh scripts/test-symbol-matrix.sh docs/cutover/symbol-matrix.tsv +git commit -S -m "docs: map Map2Check pass ABI to the core Rust API and gate it with a symbol matrix" +``` + +**Critério de conclusão:** o mapeamento cobre todos os símbolos emitidos (nenhuma linha com `-` na matriz). O script ainda falha por causa dos exports pendentes, o que é esperado até as Tasks 2.3, 2.5 e 2.8. + +--- + +## Fase 1 — Gate de toolchain (decisão D1) + +### Task 1.1: Spike de ligação do runtime Rust no pipeline LLVM 16 e no KLEE + +**Files:** +- Create (este repo): `docs/cutover/spike-llvm16.md` +- Create (este repo, descartável): `spikes/llvm16/` (não vai para `develop`; fica na branch `spike/llvm16`) + +**Interfaces:** +- Produces: a decisão D1 registrada em `docs/cutover/decisoes.md`, com a rota escolhida por gerador (`none`, `fuzzer`, `symex`) e os comandos exatos de build/link que a Task 3.1 implementa. + +- [ ] **Step 1: Montar o programa de prova** + +`spikes/llvm16/prog.c`: + +```c +extern void map2check_init(void); +extern void map2check_success(void); +extern int __VERIFIER_nondet_int(void); +int main(void) { + map2check_init(); + int x = __VERIFIER_nondet_int(); + if (x == 42) return 1; + map2check_success(); + return 0; +} +``` + +- [ ] **Step 2: Rota A (bitcode Rust no `llvm-link-16`)** + +```bash +cargo rustc --release --lib --crate-type staticlib -- --emit=llvm-bc -C panic=abort +llvm-dis-16 target/release/deps/map2check-*.bc -o /dev/null; echo "dis16=$?" +``` + +Expected (hipótese): `dis16≠0`, com o erro registrado. Registrar a mensagem exata. + +- [ ] **Step 3: Rota B (staticlib nativo no passo final)** + +```bash +cargo build --release --features libfuzzer +clang-16 -emit-llvm -c spikes/llvm16/prog.c -o prog.bc +clang-16 -fsanitize=fuzzer prog.bc target/release/libmap2check.a -ldl -lpthread -lm -o prog.fuzz +./prog.fuzz -runs=1000; echo "rc=$?" +``` + +Expected: liga e roda. Registrar se há colisão de `main`/`LLVMFuzzerTestOneInput` com a instrumentação do LibFuzzer. + +- [ ] **Step 4: Rota B sob KLEE (`--load` da biblioteca nativa)** + +```bash +cargo rustc --release --lib --crate-type cdylib -- -C panic=abort +klee --external-calls=all --load=target/release/libmap2check.so prog.bc; echo "rc=$?" +``` + +Registrar: o `__VERIFIER_nondet_int` nativo devolve um valor concreto, então o KLEE **não** explora `x == 42`. Esperado: a rota B não serve para `symex`. Confirmar com o número de caminhos (`klee-stats`). + +- [ ] **Step 5: Rota A′ (`no_std` + bitcode com rustc de LLVM ≤ 16)** + +Construir uma crate mínima `spikes/llvm16/klee-rt` com `#![no_std]`, `panic = "abort"`, um alocador global sobre `malloc`/`free` e `__VERIFIER_nondet_int`, que chama `klee_make_symbolic`. Compilar com `rustup run 1.72.1` (LLVM 16.0.x): + +```bash +rustup toolchain install 1.72.1 +cd spikes/llvm16/klee-rt && rustup run 1.72.1 cargo rustc --release -- --emit=llvm-bc +llvm-link-16 ../prog.bc target/release/deps/klee_rt-*.bc -o linked.bc && klee linked.bc +``` + +Expected: o KLEE encontra 2 caminhos. Registrar o resultado. + +- [ ] **Step 6: Rota C (custo de levar o Map2Check para LLVM 19)** + +Listar as APIs de LLVM usadas pelos passes (`grep -ohE 'llvm::[A-Za-z:]+' modules/backend/pass/*.cpp | sort -u | wc -l`) e verificar se a versão de KLEE disponível suporta LLVM 19 (documentação de release do KLEE). Registrar um sim/não e a estimativa. + +- [ ] **Step 7: Registrar a decisão** + +`docs/cutover/decisoes.md`, seção D1: rota por gerador, comandos e evidência (saídas dos Steps 2–6). **Parar aqui e pedir aprovação do mantenedor.** Se nenhuma rota servir para `symex`, o cutover completo fica bloqueado: pela restrição "não remover sem equivalente validado", o runtime C do KLEE não pode sair. O mantenedor então escolhe entre a Rota C ou adiar. + +- [ ] **Step 8: Commit (só o relatório e a decisão)** + +```bash +git add docs/cutover/spike-llvm16.md docs/cutover/decisoes.md +git commit -S -m "docs: record LLVM 16 linkage spike and runtime integration decision (D1)" +``` + +**Critério de conclusão:** D1 aprovada por escrito (comentário na issue #20), com uma rota viável para cada gerador ou com o bloqueio declarado. + +### Task 1.2: Perfil de build do runtime conforme D1 + +**Files:** +- Modify: `Cargo.toml` (perfil `runtime`, feature `klee`) +- Modify: `rust-toolchain.toml` (só se D1 = A′ exigir um toolchain separado para a crate KLEE) +- Test: `tests/runtime_profile.rs` + +**Interfaces:** +- Produces: `cargo build --profile runtime [--features libfuzzer|klee]` gera `target/runtime/libmap2check.a` com `panic=abort`. A Task 3.1 consome esse caminho. + +- [ ] **Step 1: Teste que fixa `panic = abort` no artefato** + +```rust +// SPDX-License-Identifier: GPL-3.0-only +// O runtime ligado ao programa analisado não pode desenrolar através da FFI (#8). +#[test] +fn runtime_profile_aborts_on_panic() { + let manifest = std::fs::read_to_string(concat!(env!("CARGO_MANIFEST_DIR"), "/Cargo.toml")).unwrap(); + let table = manifest.split("[profile.runtime]").nth(1).expect("perfil runtime ausente"); + let table = table.split("\n[").next().unwrap(); + assert!(table.contains("inherits = \"release\""), "{table}"); + assert!(table.contains("panic = \"abort\""), "{table}"); +} +``` + +- [ ] **Step 2: Rodar e ver falhar** + +Run: `cargo test --test runtime_profile` +Expected: FAIL, "perfil runtime ausente". + +- [ ] **Step 3: Adicionar o perfil e as features** + +```toml +[features] +libfuzzer = [] +klee = [] + +[profile.runtime] +inherits = "release" +panic = "abort" +``` + +- [ ] **Step 4: Rodar** + +Run: `cargo test --test runtime_profile && cargo build --profile runtime --features libfuzzer` +Expected: PASS e o artefato em `target/runtime/libmap2check.a`. + +- [ ] **Step 5: Commit** + +```bash +git add Cargo.toml tests/runtime_profile.rs +git commit -S -m "build: add abort-on-panic runtime profile and klee feature" +``` + +**Critério de conclusão:** o perfil existe e está testado; o CI passa. + +--- + +## Fase 2 — Paridade na biblioteca Rust (este repositório) + +Cada task desta fase fecha uma ou mais issues já abertas, que funcionam como especificação detalhada. Para cada uma, o executor cria um plano TDD próprio em `docs/superpowers/plans/` (convenção já usada neste repo) **antes** de codar. Aqui ficam fixados os testes de aceitação que precisam existir ao final. + +### Task 2.1: Correção do núcleo — #9, #8, #11, #36, #37, #38 + +**Files:** +- Modify: `rust/src/state.rs`, `rust/src/ffi.rs`, `rust/src/memtrack/mod.rs`, `rust/src/analysismode/memory.rs`, `rust/src/nondet/libfuzzer.rs` +- Test: `rust/src/ffi_tests.rs`, `tests/ffi_harness.c` + +**Interfaces:** +- Produces: `map2check_init()` passa a zerar o estado (contadores, containers, resultado e flags) em toda chamada. As Tasks 2.2–2.5 dependem disso. + +- [ ] **Step 1: Escrever os testes de aceitação (falham hoje)** + +```rust +#[test] +fn second_init_starts_from_clean_state() { // #9 + let _guard = fresh(); + map2check_init(); + unsafe { map2check_map_malloc(addr(0x1000), 8) }; + map2check_init(); + assert!(snapshot(|s| s.memtrack.is_empty())); + assert_eq!(snapshot(|s| s.current_step()), 0); +} + +#[test] +fn deref_after_free_is_invalid() { // #11 + let _guard = fresh(); + map2check_init(); + unsafe { + map2check_map_malloc(addr(0x2000), 16); + map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_check_deref(addr(0x2000), 0, 10, c"main".as_ptr()); + } + assert_violated(ViolatedProperty::MemsafetyDeref, 10, "main"); +} +``` + +Para #36, #37 e #38, usar os casos reproduzíveis de `docs/semantic-deltas.md` (SD-03, SD-06, SD-12) como testes, um `#[test]` por SD, com o nome `sd03_safe_check_keeps_earlier_violation`, `sd06_untracked_free_is_invalid` e `sd12_vla_is_not_a_leak_and_size_is_elements`. Para o #8, mais um caso em `tests/ffi_harness.c` que chame cada export antes do `map2check_init`. + +- [ ] **Step 2: Rodar e ver falhar** + +Run: `cargo test --all-features second_init deref_after_free sd03 sd06 sd12` +Expected: 5 FAIL. + +- [ ] **Step 3: Implementar seguindo o plano de cada issue** + +- [ ] **Step 4: Rodar a suíte completa e o harness** + +Run: `cargo test --all-features && cargo build --release --all-features && clang -fsanitize=address,undefined -fno-omit-frame-pointer tests/ffi_harness.c target/release/libmap2check.a -ldl -lpthread -lm -o target/ffi_harness && target/ffi_harness` +Expected: tudo verde e nenhum relatório do ASan/UBSan. + +- [ ] **Step 5: Commit por issue** (`fix: ... (#N)`), um PR por issue contra `develop`. + +**Critério de conclusão:** #8, #9, #11, #36, #37 e #38 fechadas por PRs mergeados, com os testes acima na suíte. + +### Task 2.2: Semântica de memcleanup e null-is-valid — #10 + +**Files:** +- Modify: `rust/src/ffi.rs:230-260`, `rust/src/analysismode/memory.rs`, `rust/src/state.rs` +- Test: `rust/src/ffi_tests.rs` + +**Interfaces:** +- Consumes: o reset de estado da Task 2.1. +- Produces: `AnalysisState::options { memcleanup: bool, null_is_valid: bool }`, lidas por `map2check_check_load` e `map2check_check_mem_endprog`. + +- [ ] **Step 1: Testes** + +```rust +#[test] +fn memcleanup_reports_leak_at_exit() { + let _guard = fresh(); + map2check_init(); + map2check_set_memcleanup(); + unsafe { map2check_map_malloc(addr(0x3000), 32) }; + map2check_check_mem_endprog(); + assert_eq!(snapshot(|s| s.result.property), ViolatedProperty::MemsafetyMemcleanup); +} + +#[test] +fn without_memcleanup_leak_is_memtrack() { + let _guard = fresh(); + map2check_init(); + unsafe { map2check_map_malloc(addr(0x3000), 32) }; + map2check_check_mem_endprog(); + assert_eq!(snapshot(|s| s.result.property), ViolatedProperty::MemsafetyMemtrack); +} + +#[test] +fn null_is_valid_accepts_null_load() { + let _guard = fresh(); + map2check_init(); + map2check_set_null_is_valid(); + unsafe { map2check_check_load(std::ptr::null(), 3, 0, 4, c"main".as_ptr()) }; + assert_ok(); +} +``` + +Antes de escrever o segundo teste, conferir em `AnalysisModeMemtrack.c` e `AnalysisModeMemcleanup.c` qual tag o C emite para um vazamento sem `--memcleanup-property` e ajustar a asserção ao C. Esse é o comportamento de referência. + +- [ ] **Step 2–4:** ver falhar, implementar, `cargo test --all-features` verde. +- [ ] **Step 5: Commit** `fix: honour memcleanup and null-is-valid options in memory checks (#10)` + +**Critério de conclusão:** #10 fechada; os três testes passam. + +### Task 2.3: Seleção de modo em tempo de execução, alcance de alvo e modo None + +**Files:** +- Create: `rust/src/analysismode/mode.rs` +- Modify: `rust/src/ffi.rs`, `rust/src/state.rs`, `rust/src/caller/mod.rs` (variantes `Unknown`, `Assert`, `TargetReached`) +- Test: `rust/src/ffi_tests.rs` + +**Interfaces:** +- Assinaturas atuais usadas nos testes (`rust/src/ffi.rs`): `map2check_check_deref(ptr, scope: u32, line: u32, function_name)`, `map2check_check_load(ptr, line: i32, scope: u32, size: i32, function_name)`, `map2check_map_free(var_name, ptr, scope: u32, line: u32, function_name)`, `map2check_map_malloc(ptr, size: i32)`. +- Produces: + - `#[repr(C)] pub enum AnalysisMode { None = 0, Memtrack = 1, Memcleanup = 2, Overflow = 3, Assert = 4, Target = 5 }` + - `extern "C" fn map2check_set_mode(mode: AnalysisMode)`, emitida pelo passe `Map2CheckLibrary` antes do `map2check_init` (Task 3.2). + - `pub(crate) fn mark_target_reached(func: *const c_char, scope: i32, line: i32)` (sem `exit`, testável) e `extern "C" fn map2check_reach_target(func: *const c_char, scope: i32, line: i32)`, com a mesma semântica de `map2check_target_function` no C: marca TARGET-REACHED e termina. + - `ViolatedProperty` ganha `Unknown`, `Assert` e `TargetReached` (os nomes JSON ficam definidos na Task 2.4). + +- [ ] **Step 1: Testes** + +```rust +#[test] +fn mode_none_never_reports_memory_errors() { + let _guard = fresh(); + map2check_set_mode(AnalysisMode::None); + map2check_init(); + unsafe { map2check_check_deref(addr(0xdead), 0, 1, c"main".as_ptr()) }; + assert_ok(); +} + +#[test] +fn mode_memtrack_reports_invalid_deref() { + let _guard = fresh(); + map2check_set_mode(AnalysisMode::Memtrack); + map2check_init(); + unsafe { map2check_check_deref(addr(0xdead), 0, 1, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::MemsafetyDeref, 1, "main"); +} + +#[test] +fn reach_target_sets_target_reached() { + let _guard = fresh(); + map2check_set_mode(AnalysisMode::Target); + map2check_init(); + unsafe { mark_target_reached(c"reach_error".as_ptr(), 0, 12) }; // núcleo sem exit + assert_violated(ViolatedProperty::TargetReached, 12, "reach_error"); +} + +#[test] +fn reset_keeps_selected_mode() { + let _guard = fresh(); + map2check_set_mode(AnalysisMode::Overflow); + map2check_init(); + map2check_reset(); + assert_eq!(snapshot(|s| s.mode), AnalysisMode::Overflow); +} +``` + +`mark_target_reached` é a função interna que não chama `exit`; `map2check_reach_target` = `mark_target_reached` + escrita do resultado + `exit`. Testar a versão com `exit` pelo mecanismo de processo filho já usado no #29 (`05b44c3`). + +- [ ] **Step 2–4:** ver falhar, implementar, suíte verde. A lista completa de efeitos por modo vem de `AnalysisMode*.c` (`analysis_init`, `analysis_destroy`, `analysis_is_program_correct`). Cada função vira um `match` em `mode.rs`, com um teste por braço. +- [ ] **Step 5: Commit** `feat: select analysis mode at runtime and report target reachability` + +**Critério de conclusão:** um teste por modo × função de ciclo de vida; o header regenerado traz `map2check_set_mode`, `map2check_reach_target` e `AnalysisMode`. + +### Task 2.4: Canal de resultado legado em arquivos — #19, #14 + +**Files:** +- Create: `rust/src/output/legacy_files.rs` +- Modify: `rust/src/output/mod.rs`, `rust/src/ffi.rs` (init e fim de análise) +- Test: `rust/src/output/legacy_files.rs` (módulo `tests`), `tests/legacy_golden/` (golden files) + +**Interfaces:** +- Consumes: `ViolatedProperty` com `Unknown`/`Assert`/`TargetReached` (Task 2.3). +- Produces: `pub fn write_property_files(dir: &Path, r: &AnalysisResult) -> io::Result<()>`, `pub fn write_list_log_csv`, `pub fn write_nondet_log_csv` (→ `klee_log.csv`), `pub fn write_trackbb_log` (→ `track_bb_log.st`), `pub fn write_checked_error` (→ `map2check_checked_error`) e `pub fn remove_stale_property_files(dir: &Path)`. + +- [ ] **Step 1: Capturar os golden files do C** + +Na imagem `Dockerfile.dev` com o runtime C, rodar o Map2Check em 1 tarefa por propriedade (FALSE-FREE, FALSE-DEREF, FALSE-MEMTRACK, FALSE-MEMCLEANUP, OVERFLOW, ASSERT, TARGET-REACHED, NONE) e copiar **todos** os arquivos de resultado do diretório de trabalho para `tests/legacy_golden//`. Registrar o comando exato em `tests/legacy_golden/README.md`. + +- [ ] **Step 2: Testes golden** + +```rust +#[cfg(test)] +mod tests { + use super::*; + use crate::caller::{AnalysisResult, ViolatedProperty}; + + fn golden(tag: &str, file: &str) -> String { + std::fs::read_to_string(format!( + "{}/tests/legacy_golden/{tag}/{file}", env!("CARGO_MANIFEST_DIR"))).unwrap() + } + + #[test] + fn false_deref_matches_c_runtime() { + let dir = tempfile::tempdir().unwrap(); + let r = AnalysisResult { ok: false, property: ViolatedProperty::MemsafetyDeref, + line: 7, function_name: "main".into(), ..Default::default() }; + write_property_files(dir.path(), &r).unwrap(); + assert_eq!(std::fs::read_to_string(dir.path().join("map2check_property")).unwrap(), + golden("FALSE-DEREF", "map2check_property")); + assert_eq!(std::fs::read_to_string(dir.path().join("map2check_property_deref")).unwrap(), + golden("FALSE-DEREF", "map2check_property_deref")); + } + + #[test] + fn unknown_uses_legacy_double_underscore_name() { // PropertyGenerator.c:16, ver SD novo + let dir = tempfile::tempdir().unwrap(); + let r = AnalysisResult { ok: false, property: ViolatedProperty::Unknown, ..Default::default() }; + write_property_files(dir.path(), &r).unwrap(); + assert!(dir.path().join("map2check_property__unknown").exists()); + assert!(!dir.path().join("map2check_property_unknown").exists()); + } + + #[test] + fn stale_property_files_are_removed_on_init() { + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("map2check_property_free"), "FALSE-FREE\n").unwrap(); + remove_stale_property_files(dir.path()); + assert!(!dir.path().join("map2check_property_free").exists()); + } +} +``` + +Repetir `false_deref_matches_c_runtime` como teste parametrizado para cada uma das oito tags e para `list_log.csv`, `klee_log.csv`, `track_bb_log.st` e `map2check_checked_error`. Adicionar `tempfile` a `[dev-dependencies]`. + +- [ ] **Step 3: Rodar e ver falhar.** Run: `cargo test --all-features legacy_files` → FAIL (módulo inexistente). +- [ ] **Step 4: Implementar** com escrita byte a byte igual ao golden, incluindo as quebras de linha finais. O JSON (`print_all_containers_as_json`) continua existindo; o #14 é fechado documentando em `docs/api-deltas.md` que **o canal suportado pelo Map2Check é o de arquivos** e que o JSON é um canal adicional com schema próprio, coberto por snapshot. +- [ ] **Step 5:** Registrar o SD novo em `docs/semantic-deltas.md` para o `__unknown`, com a classificação "comportamento do C preservado de propósito". +- [ ] **Step 6: Commit** `feat: write legacy property, log and witness files consumed by Map2Check frontend (#19, #14)` + +**Critério de conclusão:** os golden tests das oito tags e dos quatro logs passam; #19 e #14 estão fechadas. + +### Task 2.5: Completar a API núcleo para tudo que os passes chamam — #13, #6, #12, #7 + +Pela D5, **não há camada de compatibilidade**: nenhum nome ou assinatura do runtime C é exportado pela biblioteca. Esta task só adiciona à API núcleo, com nomes e assinaturas núcleo, as funcionalidades que ainda faltam (as linhas `owner_task` = `2.5` de `docs/cutover/abi-mapping.tsv`). A adaptação dos nomes é feita nos passes (Task 3.2). A issue #21 fecha como "won't do (D5)". + +**Files:** +- Modify: `rust/src/ffi.rs`, `rust/src/nondet/mod.rs`, `rust/src/nondet/libfuzzer.rs`, `rust/src/memtrack/mod.rs`, `docs/api-deltas.md`, `docs/cutover/abi-mapping.tsv` +- Test: `rust/src/ffi_tests.rs`, `tests/core_api_coverage.rs`, `.github/workflows/CI.yml` + +**Interfaces:** +- Consumes: `docs/cutover/abi-mapping.tsv` (Task 0.3), `map2check_set_mode`/`map2check_reach_target` (2.3), os writers (2.4) e o núcleo (2.1, 2.2). +- Produces: + - `extern "C" fn map2check_map_posix_memalign(ptr_address: *const c_void, size: i32)`, com a semântica de `map2check_posix` (`AnalysisModeMemtrack.c:328`). + - `__VERIFIER_nondet_{loff_t,sector_t,pchar}` e `map2check_save_nondet_log_` para todo tipo listado no mapeamento (#6), com a representação C de cada tipo confirmada no `NonDetGenerator.h` do Map2Check. + - A decisão do #13 registrada em `docs/api-deltas.md`: a assinatura de `map2check_save_nondet_log_*` continua a núcleo (valor por parâmetro, retorno `void`), e o passe se adapta. + - Ao final, todo `core_symbol` do mapeamento existe no header. A Task 3.2 depende de `scripts/symbol-matrix.sh` sair com 0. + +- [ ] **Step 1: Teste que exige cada alvo do mapeamento no artefato** + +```rust +// SPDX-License-Identifier: GPL-3.0-only +// Todo símbolo núcleo usado pelo mapeamento ABI C → núcleo precisa existir no staticlib. +// Não há camada de compatibilidade (D5): nomes do runtime C não podem aparecer no artefato. +fn defined_symbols() -> String { + let root = env!("CARGO_MANIFEST_DIR"); + let status = std::process::Command::new("cargo") + .args(["build", "--profile", "runtime", "--features", "libfuzzer"]) + .current_dir(root).status().unwrap(); + assert!(status.success()); + let nm = std::process::Command::new("nm") + .args(["--defined-only", "-g", &format!("{root}/target/runtime/libmap2check.a")]) + .output().unwrap(); + String::from_utf8_lossy(&nm.stdout).into_owned() +} + +fn mapping() -> Vec<(String, String)> { + let root = env!("CARGO_MANIFEST_DIR"); + std::fs::read_to_string(format!("{root}/docs/cutover/abi-mapping.tsv")).unwrap() + .lines().skip(1) + .map(|l| { let mut c = l.split('\t'); (c.next().unwrap().into(), c.next().unwrap().into()) }) + .collect() +} + +fn is_defined(defined: &str, sym: &str) -> bool { + defined.lines().any(|d| d.ends_with(&format!(" T {sym}"))) +} + +#[test] +fn every_core_target_is_exported() { + let defined = defined_symbols(); + let missing: Vec<_> = mapping().iter() + .flat_map(|(_, core)| core.split('+').map(str::to_owned).collect::>()) + .filter(|c| c != "-" && c != "__map2check_main__" && !is_defined(&defined, c)) + .collect(); + assert!(missing.is_empty(), "alvos núcleo ausentes: {missing:?}"); +} + +#[test] +fn no_legacy_c_name_is_exported() { + let defined = defined_symbols(); + let leaked: Vec<_> = mapping().into_iter() + .filter(|(c, core)| !core.split('+').any(|x| x == c) && is_defined(&defined, c)) + .map(|(c, _)| c) + .collect(); + assert!(leaked.is_empty(), "nomes do runtime C exportados (D5 proíbe): {leaked:?}"); +} +``` + +- [ ] **Step 2: Rodar e ver falhar.** Run: `cargo test --test core_api_coverage` → FAIL em `every_core_target_is_exported`, listando os exports das linhas `owner_task` = `2.5`. O `no_legacy_c_name_is_exported` já deve passar. +- [ ] **Step 3: Implementar um export por commit pequeno**, cada um com um teste em `ffi_tests.rs` que o chame como o C faria e verifique o efeito no estado. Por exemplo: + +```rust +#[test] +fn posix_memalign_is_tracked_like_malloc() { + let _guard = fresh(); + map2check_init(); + unsafe { map2check_map_posix_memalign(addr(0x4000), 64) }; + assert!(snapshot(|s| !s.memtrack.is_empty())); +} +``` + +- [ ] **Step 4: Rodar.** Run: `cargo test --all-features && scripts/symbol-matrix.sh ~/Documents/Dev/Map2Check docs/cutover/abi-mapping.tsv >/dev/null; echo $?` → `0` (com 2.3 e 2.8 já feitas). +- [ ] **Step 5: Ligar os três programas C de `test/` e o harness à biblioteca** (#7), atualizando-os para a API núcleo onde ainda usarem nomes antigos. Adicionar os passos de compilação e execução ao `.github/workflows/CI.yml`. +- [ ] **Step 6: Commit/PR** `feat: complete the core C API for every Map2Check instrumentation point (#13, #6, #12, #7)`. Fechar a #21 com o comentário "won't do: D5, sem camada de compatibilidade; ver docs/cutover/decisoes.md". + +**Critério de conclusão:** `core_api_coverage` (os dois testes) e `ffi_harness` passam no CI, e `symbol-matrix.sh` sai com 0 contra o `develop` atual do Map2Check. + +### Task 2.6: Estrutura de dados `--btree` (decisão D3) + +**Files:** `docs/cutover/decisoes.md` (D3); se a decisão for portar: `rust/src/state.rs` + backend de armazenamento. + +- [ ] **Step 1:** Rodar a Task 4.3 (medição de memória) antes desta. +- [ ] **Step 2:** Se o pico de RSS do Rust em memória ficar ≤ o do C com `--btree` em todas as tarefas do corpus e nas 5 maiores do `MemSafety-Arrays.set`, registrar a justificativa em D3. A Task 3.3 então trata `--btree` como no-op com aviso (`Log::Warning("--btree is ignored by the Rust runtime")`), e um teste no frontend fixa o aviso. +- [ ] **Step 3:** Caso contrário, abrir um plano próprio para portar o armazenamento em disco e só seguir para a Fase 5 com ele pronto. + +**Critério de conclusão:** D3 registrada com números, e a implementação ou o no-op testado. + +### Task 2.7: Gerador KLEE — #20 + +**Files:** conforme D1. Na rota A′: `klee-rt/` (crate `no_std` separada no workspace) + `docs/klee-feasibility.md`. + +**Interfaces:** +- Produces: um `.bc` legível por `llvm-dis-16` que exporta `__VERIFIER_nondet_*` / `map2check_non_det_*` via `klee_make_symbolic` e `map2check_assume` via `klee_assume`, mais as funções de estado necessárias ao modo, com a mesma semântica do núcleo. + +- [ ] **Step 1:** Escrever o teste de integração `tests/klee_paths.sh`, que liga `spikes/llvm16/prog.c` ao `.bc` e espera `KLEE: done: completed paths = 2`. +- [ ] **Step 2:** Ver falhar. Implementar conforme D1. +- [ ] **Step 3:** Ver passar no container `Dockerfile.dev`. +- [ ] **Step 4:** Commit/PR `feat: add KLEE nondet runtime (#20)`. + +**Critério de conclusão:** `tests/klee_paths.sh` passa no CI do Map2Check (Task 3.4), e #20 está fechada com `docs/klee-feasibility.md`. + +### Task 2.8: Runtime wasm (decisão D2) + +**Files:** `rust/src/wasm.rs` (feature `wasm`), testes em `rust/src/ffi_tests.rs`. + +**Interfaces:** +- Produces: `wasm_rt_allocate_memory`, `wasm_rt_grow_memory`, `wasm_rt_free_memory`, `wasm_rt_allocate_funcref_table`, `wasm_rt_free_funcref_table`, `w2c_wasi__snapshot__preview1_proc_exit`, `wasm_rt_is_initialized`, `wasm_rt_init`, `wasm_rt_free`, `wasm_rt_trap`, `map2check_wasm_malloc`, `map2check_wasm_free` e `map2check_wasm_check_access`, com o layout de `wasm_rt_memory_t` do wabt 1.0.41 replicado via `#[repr(C)]` e conferido por um teste de `size_of`/`offset_of` contra o `wasm-rt.h` compilado em `build.rs` (apenas quando `WASM_RT_INCLUDE` estiver definido). + +- [ ] **Step 1:** Escrever o teste de layout e um teste por função, espelhando `WasmRuntimeStubs.c:22-110`. +- [ ] **Step 2–4:** Ver falhar, implementar, ver passar. +- [ ] **Step 5:** Commit `feat: port wasm2c runtime stubs to Rust`. + +**Critério de conclusão:** com o runtime Rust, `tests/integration/test_wasm_pipeline.sh` e `test_wasm_entrypoint.sh` do Map2Check passam (verificado na Task 4.2). + +### Task 2.9: Release candidata da biblioteca + +- [ ] **Step 1:** Subir a versão para `1.1.0-rc1` em `Cargo.toml` e atualizar o `CHANGELOG.md` com as seções das Tasks 2.1–2.8. +- [ ] **Step 2:** Run: `cargo test --all-features && cargo clippy --all-targets --all-features -- -D warnings && cargo fmt --check && cargo deny check && scripts/coverage.sh`. Expected: tudo verde. +- [ ] **Step 3:** Tag assinada `v1.1.0-rc1` em `main` (fluxo de [docs/release.md](../../release.md)). + +**Critério de conclusão:** o artefato publicado inclui `libmap2check.a` (perfil `runtime`) e, se D1 exigir, o `.bc` KLEE. + +--- + +## Fase 3 — Integração no Map2Check por migração direta, em branch (#22, #23) + +**Migração direta (decidida em 2026-10-15):** os passes passam a emitir só a API núcleo, sem tabela de ABI dupla e sem opção de build para voltar ao C. Todo o trabalho desta fase vai para uma branch de longa duração, `feat/rust-runtime`, no Map2Check, criada a partir de `develop`, rebaseada semanalmente e mergeada **em um único merge commit** só depois do gate da Fase 4 (Task 5.1). Até lá, `develop` continua com o runtime C intacto. Depois do merge, o rollback é `git revert -m 1 ` (Task 5.1), possível até a Fase 6. + +### Task 3.1: CMake constrói e instala o runtime Rust + +**Files:** +- Create (Map2Check): `cmake/Map2CheckRustRuntime.cmake` +- Modify (Map2Check): `CMakeLists.txt`, `modules/backend/CMakeLists.txt` +- Test (Map2Check): `tests/cutover/test_rust_runtime_cmake.sh` + +**Interfaces:** +- Consumes: `v1.1.0-rc1` da biblioteca (Task 2.9). +- Produces: o build instala `${MAP2CHECK_PATH}/lib/libmap2check.a` (e `map2check-klee.bc`, conforme D1) e **não** compila nem instala os `.bc` de `modules/backend/library`. O diretório C continua no repositório (só sai na Fase 6), mas fora do build. Variáveis de cache: `MAP2CHECK_RUST_LIB_DIR` (biblioteca pré-compilada) e `MAP2CHECK_RUST_SRC` (checkout para compilar com cargo). + +- [ ] **Step 1: Teste** + +```bash +#!/usr/bin/env bash +# Configura, instala e verifica que só o runtime Rust é instalado. +set -euo pipefail +ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +B="$(mktemp -d)"; P="$(mktemp -d)"; trap 'rm -rf "$B" "$P"' EXIT +cmake -S "$ROOT" -B "$B" -DMAP2CHECK_RUST_LIB_DIR="${MAP2CHECK_RUST_LIB_DIR:?}" >/dev/null +cmake --build "$B" -j >/dev/null && cmake --install "$B" --prefix "$P" >/dev/null +test -f "$P/lib/libmap2check.a" +if ls "$P"/lib/*.bc 2>/dev/null | grep -vE 'map2check-klee\.bc$'; then + echo "runtime C ainda instalado" >&2; exit 1 +fi +# Diretório sem a biblioteca: o configure tem que falhar alto. +E="$(mktemp -d)" +if cmake -S "$ROOT" -B "$B.bad" -DMAP2CHECK_RUST_LIB_DIR="$E" >/dev/null 2>&1; then + echo "configure deveria falhar sem libmap2check.a" >&2; exit 1 +fi +rm -rf "$E" "$B.bad" +echo ok +``` + +- [ ] **Step 2: Ver falhar.** Expected: FAIL em `test -f "$P/lib/libmap2check.a"`. +- [ ] **Step 3: Implementar `cmake/Map2CheckRustRuntime.cmake`** + +```cmake +# Runtime do Map2Check: map2check-library (Rust). Substitui modules/backend/library. +set(MAP2CHECK_RUST_LIB_DIR "" CACHE PATH "Directory with a prebuilt libmap2check.a (empty = build from MAP2CHECK_RUST_SRC)") +set(MAP2CHECK_RUST_SRC "" CACHE PATH "map2check-library checkout used when MAP2CHECK_RUST_LIB_DIR is empty") + +if(MAP2CHECK_RUST_LIB_DIR) + set(_m2c_rust_lib "${MAP2CHECK_RUST_LIB_DIR}/libmap2check.a") + if(NOT EXISTS "${_m2c_rust_lib}") + message(FATAL_ERROR "libmap2check.a not found in ${MAP2CHECK_RUST_LIB_DIR}") + endif() +elseif(MAP2CHECK_RUST_SRC) + find_program(CARGO cargo REQUIRED) + set(_m2c_rust_lib "${MAP2CHECK_RUST_SRC}/target/runtime/libmap2check.a") + add_custom_command(OUTPUT "${_m2c_rust_lib}" + COMMAND ${CARGO} build --profile runtime --features libfuzzer + WORKING_DIRECTORY "${MAP2CHECK_RUST_SRC}") + add_custom_target(Map2CheckRustRuntime ALL DEPENDS "${_m2c_rust_lib}") +else() + message(FATAL_ERROR "Set MAP2CHECK_RUST_LIB_DIR or MAP2CHECK_RUST_SRC to provide the Rust runtime") +endif() +install(FILES "${_m2c_rust_lib}" DESTINATION lib) +``` + +Em `modules/backend/CMakeLists.txt`, remover `add_subdirectory(library)`. Se D1 incluir o `.bc` KLEE, adicionar a verificação `execute_process(COMMAND llvm-dis-16 -o /dev/null RESULT_VARIABLE r)` + `FATAL_ERROR` quando `r` ≠ 0 (Review Focus 5). + +- [ ] **Step 4: Ver passar.** Run: `MAP2CHECK_RUST_LIB_DIR=~/Documents/Dev/map2check-library/target/runtime bash tests/cutover/test_rust_runtime_cmake.sh` → `ok`. +- [ ] **Step 5: Commit (na branch `feat/rust-runtime`)** `build: build and install the Rust runtime instead of the C bitcode library` + +**Critério de conclusão:** o teste passa, e o configure falha com mensagem clara sem a biblioteca. + +### Task 3.2: Passes LLVM emitem a API núcleo + +Pela D5, a biblioteca não se adapta: os passes passam a emitir os nomes e assinaturas núcleo de `docs/cutover/abi-mapping.tsv`. Os nomes ficam centralizados em uma única tabela, para que exista um só lugar a revisar contra o mapeamento. + +**Files:** +- Create (Map2Check): `modules/backend/pass/RuntimeABI.hpp`, `modules/backend/pass/RuntimeABI.cpp` +- Modify (Map2Check): `AssertPass.cpp`, `OverflowPass.cpp`, `MemoryTrackPass.cpp`, `NonDetPass.cpp`, `NonDetFunctions.hpp`, `TargetPass.cpp`, `TrackBasicBlockPass.cpp`, `LoopPredAssumePass.cpp`, `Map2CheckLibrary.cpp`, `LibraryFunctions.hpp`, `OperationsFunctions.hpp` +- Test (Map2Check): `tests/unit/pass/RuntimeABITest.cpp`, `tests/unit/pass/PassIRTest.cpp`, `tests/unit/pass/inputs/*.c`, `tests/unit/pass/CMakeLists.txt` + +**Interfaces:** +- Consumes: `docs/cutover/abi-mapping.tsv` e a biblioteca com todos os alvos exportados (Task 2.5); `AnalysisMode` com os valores numéricos None=0, Memtrack=1, Memcleanup=2, Overflow=3, Assert=4, Target=5 (Task 2.3). +- Produces: + +```cpp +// RuntimeABI.hpp +namespace Map2Check { +enum class RuntimeHook { + SetMode, Init, Success, TrackBB, TargetReached, Assert, Assume, + BinopAdd, BinopAddU, BinopSub, BinopSubU, BinopMul, BinopMulU, BinopSDiv, + Malloc, Calloc, Alloca, NonStaticAlloca, FunctionAddr, StorePointer, + Load, CheckFree, MapFree, FreeResolved, PosixMemalign, + WasmMalloc, WasmFree, WasmCheckAccess +}; +// Uma função de runtime da API núcleo: nome + tipo, para getOrInsertFunction. +struct RuntimeCall { const char* name; llvm::FunctionType* type; }; +RuntimeCall runtimeCall(RuntimeHook hook, llvm::LLVMContext& ctx); +std::string nonDetGenerator(llvm::StringRef cType); // __VERIFIER_nondet_ +std::string nonDetLogger(llvm::StringRef cType); // map2check_save_nondet_log_ +} // namespace Map2Check +``` + + Cada passe troca `getOrInsertFunction("map2check_…", …)` por `runtimeCall(RuntimeHook::…, ctx)` e monta os argumentos na ordem núcleo, seguindo as notas do mapeamento (ex.: `Load` passa linha, escopo, tamanho e função; o `free` do C vira `CheckFree` + `MapFree`; `assume_loop`/`crab_assume` fazem `zext` para `i32` e chamam `Assume`; `init` deixa de passar o `isSvComp` morto). `Map2CheckLibrary.cpp` emite `SetMode()` antes de `Init`. + +- [ ] **Step 1: Teste da tabela contra o mapeamento** + +```cpp +// Os nomes da tabela batem com docs/cutover/abi-mapping.tsv, e nenhum nome do runtime C sobrou. +TEST(RuntimeABI, NamesMatchMapping) { + llvm::LLVMContext ctx; + std::set core = loadCoreSymbols(MAP2CHECK_ABI_MAPPING_TSV); // coluna core_symbol, separada por '+' + for (int h = 0; h <= static_cast(RuntimeHook::WasmCheckAccess); ++h) + EXPECT_TRUE(core.count(runtimeCall(static_cast(h), ctx).name)) + << runtimeCall(static_cast(h), ctx).name; +} + +TEST(RuntimeABI, NoLegacyNameRemains) { + std::set legacy = loadLegacyOnlySymbols(MAP2CHECK_ABI_MAPPING_TSV); // c_symbol que não é também core + llvm::LLVMContext ctx; + for (int h = 0; h <= static_cast(RuntimeHook::WasmCheckAccess); ++h) + EXPECT_FALSE(legacy.count(runtimeCall(static_cast(h), ctx).name)); +} +``` + +`MAP2CHECK_ABI_MAPPING_TSV` é uma definição do CMake de teste que aponta para uma cópia do `abi-mapping.tsv` baixada junto com o tarball da tag `v1.1.0-rc1` da biblioteca (Task 3.4). + +- [ ] **Step 2: Teste de IR por hook.** Para cada hook, um programa C mínimo em `tests/unit/pass/inputs/` que exercita o ponto de instrumentação (ex.: `free.c` com `free(p)`). Compilar com `clang-16 -O0 -g -emit-llvm`, rodar o passe com `opt-16 -load-pass-plugin`, executar `llvm::verifyModule` e conferir os callees: + +```cpp +TEST(PassIR, MemoryTrackFreeEmitsCoreCalls) { + auto m = runPassOn("MemoryTrackPass", "free.c"); + ASSERT_FALSE(llvm::verifyModule(*m, &llvm::errs())); + EXPECT_TRUE(calls(*m, "map2check_check_free")); + EXPECT_TRUE(calls(*m, "map2check_map_free")); + EXPECT_FALSE(calls(*m, "map2check_free")); +} + +TEST(PassIR, LibraryEmitsSetModeBeforeInit) { + auto m = runPassOn("Map2CheckLibrary", "main.c", /*mode=*/"memtrack"); + ASSERT_FALSE(llvm::verifyModule(*m, &llvm::errs())); + EXPECT_LT(callIndex(*m, "map2check_set_mode"), callIndex(*m, "map2check_init")); + EXPECT_EQ(firstArgConstant(*m, "map2check_set_mode"), 1); // Memtrack +} +``` + +Escrever um teste por hook do enum (28), mais `nonDetGenerator`/`nonDetLogger` para cada tipo do mapeamento. + +- [ ] **Step 3: Rodar e ver falhar.** Run: `cmake --build build && cd build && ctest -R 'RuntimeABI|PassIR' --output-on-failure` → falha de compilação (`RuntimeABI.hpp` inexistente). +- [ ] **Step 4: Implementar a tabela e migrar um passe por commit**, na ordem `TrackBasicBlockPass`, `AssertPass`, `TargetPass`, `LoopPredAssumePass`, `OverflowPass`, `NonDetPass`, `MemoryTrackPass`, `Map2CheckLibrary`. Cada commit deixa os testes `PassIR` do passe migrado verdes. +- [ ] **Step 5: Teste de link ponta a ponta.** Para cada input de `tests/unit/pass/inputs/`, instrumentar com todos os passes do modo, ligar a `libmap2check.a` com `clang-16` e garantir zero `undefined reference`. +- [ ] **Step 6: Commit (na branch)** `feat(pass): emit the core runtime API of map2check-library (#23)` + +**Critério de conclusão:** `RuntimeABI*` e `PassIR*` passam; `grep -rnE '"map2check_(malloc|calloc|free|load|alloca|assert|track_bb|function|posix|binop_(add|sub|mul|sdiv)(_uint)?|non_det_[a-z_]+|nondet_[a-z_]+)"' modules/backend/pass` não retorna nada; o link ponta a ponta não tem símbolos indefinidos. + +### Task 3.3: Link do runtime Rust em `Caller::linkLLVM` + +**Files:** +- Modify (Map2Check): `modules/frontend/caller.cpp:226-312` e os pontos que geram os binários `-fuzzed.out` (`:121-133`) e o comando KLEE (`:384-420`) +- Test (Map2Check): `tests/unit/frontend/CallerLinkTest.cpp` (gtest; o diretório já tem a infraestrutura do `ctest`) + +**Interfaces:** +- Consumes: os passes emitindo a API núcleo (Task 3.2) e o `libmap2check.a` instalado (Task 3.1). +- Produces: `linkLLVM()` não liga nenhum `.bc` de runtime; os comandos de `clang` final acrescentam `${MAP2CHECK_PATH}/lib/libmap2check.a -ldl -lpthread -lm`. No KLEE, segue a rota D1. As funções puras `std::string buildLinkCommand(const Caller::LinkInputs&)` e `std::string buildFuzzerCommand(const Caller::LinkInputs&)` ficam testáveis. + +- [ ] **Step 1: Teste do comando gerado** + +```cpp +TEST(CallerLink, LinksOnlyTheRustRuntime) { + Caller::LinkInputs in{"h", Map2CheckMode::MEMTRACK_MODE, NonDetGenerator::LibFuzzer, + DataStructure::Array, /*wasm=*/false, /*witness=*/false}; + std::string link = buildLinkCommand(in); + EXPECT_EQ(link.find("${MAP2CHECK_PATH}/lib/"), std::string::npos); + EXPECT_NE(buildFuzzerCommand(in).find("libmap2check.a"), std::string::npos); +} + +TEST(CallerLink, EveryModeAndGeneratorLinksRuntime) { + for (auto mode : {Map2CheckMode::MEMTRACK_MODE, Map2CheckMode::MEMCLEANUP_MODE, + Map2CheckMode::OVERFLOW_MODE, Map2CheckMode::ASSERT_MODE, + Map2CheckMode::REACHABILITY_MODE}) + for (auto gen : {NonDetGenerator::LibFuzzer, NonDetGenerator::None}) { + Caller::LinkInputs in{"h", mode, gen, DataStructure::Array, false, false}; + EXPECT_NE(buildFuzzerCommand(in).find("libmap2check.a"), std::string::npos); + } +} +``` + +Conferir os nomes exatos dos enumeradores em `modules/frontend/caller.hpp` antes de escrever o teste. Os nomes acima seguem `caller.cpp:254-280` e devem ser ajustados se diferirem. + +- [ ] **Step 2–4:** Ver falhar, extrair as funções e implementar, `ctest --output-on-failure` verde. +- [ ] **Step 5:** Se D3 = remover, adicionar o aviso de `--btree` com o seu teste. +- [ ] **Step 6: Commit (na branch)** `feat: link map2check-library instead of the C bitcode runtime` + +**Critério de conclusão:** na branch, `map2check --memtrack` sobre `release/test_heap_overflow.c` imprime `VERIFICATION FAILED` e grava `map2check_property` com `FALSE-DEREF`. + +### Task 3.4: CI do Map2Check na branch + +**Files:** Modify (Map2Check): `.github/workflows/ci.yml`, `Dockerfile.dev` + +- [ ] **Step 1:** Em `Dockerfile.dev`, baixar o tarball do release `v1.1.0-rc1` da biblioteca (gerado por `scripts/package-release.sh`), com SHA-256 fixado, extrair em `/opt/map2check-library` e passar `-DMAP2CHECK_RUST_LIB_DIR=/opt/map2check-library/lib` ao CMake. Fazer o mesmo nos jobs de build, `ctest`, `e2e-wasm` e Valgrind do `ci.yml`. +- [ ] **Step 2:** Remover do CI, **nesta branch**, os passos que compilam ou analisam o runtime C (`cppcheck` do backend C, os `.bc` e `tests/unit/backend-library` do `ctest`), já que ele saiu do build. O código continua no repositório até a Fase 6. +- [ ] **Step 3:** Abrir o PR draft `feat/rust-runtime` → `develop` e confirmar os jobs verdes. O PR fica em draft até a Task 5.1. + +**Critério de conclusão:** CI verde na branch com as mesmas dependências externas (wabt 1.0.41, klee-uclibc, z3/stp) de `develop`. + +--- + +## Fase 4 — Validação diferencial (#15, #18) + +### Task 4.1: Comparador de linhas de base + +**Files:** +- Create (Map2Check): `scripts/cutover/compare.py` +- Create (Map2Check): `scripts/cutover/test_compare.py` +- Create (Map2Check): `tests/cutover/deltas-aceitos.tsv` (cabeçalho `task prp gen field c_value rust_value sd_ref`) + +**Interfaces:** +- Consumes: dois TSVs no formato da Task 0.2. +- Produces: `compare.py --accepted deltas-aceitos.tsv [--max-wall 1.10] [--max-rss 1.25]`, que sai com 0 quando os critérios 1–8 passam e com 1 caso contrário, e imprime um relatório em markdown no stdout. + +- [ ] **Step 1: Testes** + +```python +# Testes do comparador do gate de cutover. +import subprocess, sys, textwrap, pathlib, tempfile + +HERE = pathlib.Path(__file__).parent +HDR = "task\tprp\tgen\trep\tverdict\ttag\tline\tfunction\twitness\twall_s\tmaxrss_kb\n" + +def run(c_rows, r_rows, accepted=""): + d = pathlib.Path(tempfile.mkdtemp()) + (d / "c.tsv").write_text(HDR + "".join(c_rows)) + (d / "r.tsv").write_text(HDR + "".join(r_rows)) + (d / "a.tsv").write_text("task\tprp\tgen\tfield\tc_value\trust_value\tsd_ref\n" + accepted) + return subprocess.run([sys.executable, HERE / "compare.py", d / "c.tsv", d / "r.tsv", + "--accepted", d / "a.tsv"], capture_output=True, text=True) + +ROW = "t.yml\tvalid-memsafety\tfuzzer\t{rep}\t{v}\t{tag}\t7\tmain\t0\t{wall}\t1000\n" + +def rows(v="FAILED", tag="FALSE-DEREF", wall=1.0): + return [ROW.format(rep=i, v=v, tag=tag, wall=wall) for i in range(1, 6)] + +def test_identical_passes(): + assert run(rows(), rows()).returncode == 0 + +def test_verdict_change_blocks(): + p = run(rows(), rows(v="UNKNOWN", tag="-")) + assert p.returncode == 1 and "verdict" in p.stdout + +def test_accepted_delta_passes(): + acc = "t.yml\tvalid-memsafety\tfuzzer\ttag\tFALSE-DEREF\tFALSE-FREE\tSD-06\n" + assert run(rows(), rows(tag="FALSE-FREE"), acc).returncode == 0 + +def test_slowdown_over_10_percent_blocks(): + p = run(rows(wall=1.0), rows(wall=1.2)) + assert p.returncode == 1 and "wall" in p.stdout + +def test_new_timeout_blocks(): + p = run(rows(), rows(v="TIMEOUT", tag="-")) + assert p.returncode == 1 and "TIMEOUT" in p.stdout +``` + +- [ ] **Step 2: Ver falhar.** Run: `python3 -m pytest scripts/cutover/test_compare.py -q` → 5 erros (o arquivo não existe). +- [ ] **Step 3: Implementar** + +```python +#!/usr/bin/env python3 +"""Gate diferencial do cutover: compara as linhas de base C e Rust (critérios 1-8).""" +import argparse, csv, statistics, sys +from collections import Counter, defaultdict + +KEY = ("task", "prp", "gen") +FIELDS = ("verdict", "tag", "line", "function", "witness") + + +def load(path): + groups = defaultdict(list) + with open(path, newline="") as f: + for row in csv.DictReader(f, delimiter="\t"): + groups[tuple(row[k] for k in KEY)].append(row) + return groups + + +def mode(rows, field): + return Counter(r[field] for r in rows).most_common(1)[0][0] + + +def median_num(rows, field): + vals = [float(r[field]) for r in rows if r[field] not in ("-", "")] + return statistics.median(vals) if vals else None + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("c"); ap.add_argument("rust") + ap.add_argument("--accepted", required=True) + ap.add_argument("--max-wall", type=float, default=1.10) + ap.add_argument("--max-rss", type=float, default=1.25) + a = ap.parse_args() + c, r = load(a.c), load(a.rust) + accepted = set() + with open(a.accepted, newline="") as f: + for row in csv.DictReader(f, delimiter="\t"): + accepted.add((row["task"], row["prp"], row["gen"], row["field"], row["c_value"], row["rust_value"])) + + failures, wall_ratios, rss_ratios = [], [], [] + timeouts = Counter() + for key in sorted(set(c) | set(r)): + if key not in r or key not in c: + failures.append(f"| {' '.join(key)} | missing | {key in c} | {key in r} |") + continue + for field in FIELDS: + cv, rv = mode(c[key], field), mode(r[key], field) + if cv != rv and (*key, field, cv, rv) not in accepted: + failures.append(f"| {' '.join(key)} | {field} | {cv} | {rv} |") + for side, rows in (("c", c[key]), ("rust", r[key])): + if mode(rows, "verdict") == "TIMEOUT": + timeouts[(key[2], side)] += 1 + cw, rw = median_num(c[key], "wall_s"), median_num(r[key], "wall_s") + if cw and rw: + wall_ratios.append(rw / cw) + cm, rm = median_num(c[key], "maxrss_kb"), median_num(r[key], "maxrss_kb") + if cm and rm: + rss_ratios.append(rm / cm) + + for gen in {k[2] for k in c}: + if timeouts[(gen, "rust")] > timeouts[(gen, "c")]: + failures.append(f"| gen={gen} | TIMEOUT count | {timeouts[(gen, 'c')]} | {timeouts[(gen, 'rust')]} |") + wall = statistics.median(wall_ratios) if wall_ratios else 1.0 + rss = statistics.median(rss_ratios) if rss_ratios else 1.0 + if wall > a.max_wall: + failures.append(f"| all | median wall ratio | <= {a.max_wall} | {wall:.3f} |") + if rss > a.max_rss: + failures.append(f"| all | median rss ratio | <= {a.max_rss} | {rss:.3f} |") + + print(f"median wall ratio: {wall:.3f} median rss ratio: {rss:.3f}\n") + if failures: + print("| key | field | c | rust |\n|---|---|---|---|") + print("\n".join(failures)) + return 1 + print("gate: PASS") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) +``` + +- [ ] **Step 4: Ver passar.** Run: `python3 -m pytest scripts/cutover/test_compare.py -q` → `5 passed`. +- [ ] **Step 5: Commit** `test: add differential gate comparing C and Rust runtime baselines` + +**Critério de conclusão:** os cinco testes passam e o relatório é legível no log de CI. + +### Task 4.2: Gate diferencial completo + +- [ ] **Step 1:** Gerar `tests/cutover/baseline-rust.tsv` a partir da branch `feat/rust-runtime` e regenerar `baseline-c.tsv` a partir do merge-base da branch com `develop`, na mesma imagem e máquina da Task 0.2. Assim os dois lados veem o mesmo frontend, a menos das mudanças da Fase 3. +- [ ] **Step 2:** Run: `python3 scripts/cutover/compare.py tests/cutover/baseline-c.tsv tests/cutover/baseline-rust.tsv --accepted tests/cutover/deltas-aceitos.tsv`. +- [ ] **Step 3:** Para cada linha de falha: (a) bug do Rust → issue + correção na Fase 2 + repetir; (b) bug do C → SD novo em `docs/semantic-deltas.md` com caso reproduzível + linha em `deltas-aceitos.tsv`, **aprovada pelo mantenedor no PR**. Nunca aceitar uma divergência sem um SD. +- [ ] **Step 4:** Rodar também `tests/integration/test_wasm_*.sh` com o runtime Rust (D2) e `ctest -T memcheck` (Valgrind) com o runtime Rust. +- [ ] **Step 5:** Adicionar um job agendado semanal (`schedule: cron`) que roda as Tasks 0.2 e 4.1 com `REPS=3` e publica o relatório como artefato do CI (#18). +- [ ] **Step 6:** Commit das linhas de base e dos deltas aceitos. + +**Critério de conclusão:** `gate: PASS` em duas execuções consecutivas (critério de prontidão), registradas nos PRs com os links dos jobs. + +### Task 4.3: Medição de memória para D3 e D4 + +- [ ] **Step 1:** Rodar `run-corpus.sh` com o runtime C e `--btree`. Para isso, acrescentar temporariamente `--btree` a `flags_for` via a variável `EXTRA_FLAGS`, adicionada ao script com o teste correspondente em `test-run-corpus.sh`. Usar o corpus regular e as 5 maiores tarefas (por linhas) de `MemSafety-Arrays.set`. +- [ ] **Step 2:** Comparar o pico de RSS: C+BTree × C+Array × Rust. +- [ ] **Step 3:** Registrar a tabela em `docs/cutover/perf.md` e as decisões D3/D4 em `docs/cutover/decisoes.md`, com aprovação do mantenedor. + +**Critério de conclusão:** D3 e D4 têm números e aprovação. + +--- + +## Fase 5 — Merge da migração (reversível por revert) + +### Task 5.1: Mergear `feat/rust-runtime` + +- [ ] **Step 1:** Rebasear a branch sobre `develop`, regenerar as duas linhas de base no mesmo commit-base (C a partir do merge-base, Rust a partir da branch; Task 4.2, Step 1) e confirmar `gate: PASS` pela segunda vez consecutiva (critério de prontidão). +- [ ] **Step 2:** Ensaiar o rollback antes do merge. Em um clone descartável, fazer o merge local, depois `git revert -m 1 HEAD`, build e `ctest`. Registrar no PR que o revert restaura o runtime C e passa. +- [ ] **Step 3:** README do Map2Check: na seção de build, documentar `MAP2CHECK_RUST_LIB_DIR`/`MAP2CHECK_RUST_SRC` e a versão mínima da biblioteca. CHANGELOG: entrada "Runtime replaced by map2check-library (Rust)". +- [ ] **Step 4:** Tirar o PR de draft e mergear com **merge commit** (não squash, não rebase), para que um único `git revert -m 1` desfaça tudo. Anotar o hash do merge no PR e em `docs/cutover/decisoes.md`. +- [ ] **Step 5:** Publicar uma release do Map2Check. +- [ ] **Step 6:** Período de observação de **um ciclo de release**. Toda regressão reportada é triada; se for de runtime e não tiver correção rápida, o rollback é `git revert -m 1 ` em `develop` + issue + nova release. + +**Critério de conclusão:** uma release publicada com o runtime Rust, nenhum revert acionado e o gate 4.2 verde de novo ao fim do ciclo. + +### Task 5.2: Portar as asserções dos testes unitários C + +**Files:** ler `Map2Check/tests/unit/backend-library/{AllocationLog,BTree,Container,HeapLog,MemTrack}Test.cpp`; criar o equivalente em `rust/src/ffi_tests.rs`. + +- [ ] **Step 1:** Para cada `TEST(...)`/`EXPECT_*`, escrever um `#[test]` com o mesmo nome em snake_case e a mesma asserção sobre o export núcleo equivalente (segundo `docs/cutover/abi-mapping.tsv`). Os testes de `BTree` só migram se D3 = portar; se não, cada um é listado em `docs/cutover/decisoes.md` (D3) como coberto pela decisão. +- [ ] **Step 2:** Rodar `cargo test --all-features`. +- [ ] **Step 3:** Commit `test: port Map2Check C runtime unit assertions to Rust`. + +**Critério de conclusão:** a tabela de correspondência `TEST C → #[test] Rust` está em `docs/cutover/decisoes.md` e todos passam. + +--- + +## Fase 6 — Remoção do C (contém o PONTO DE NÃO-RETORNO) + +**Ponto de não-retorno:** o merge, em `develop` do Map2Check, do PR da Task 6.2. Antes dele, o runtime C ainda está no repositório (fora do build) e o merge da Task 5.1 pode ser revertido com `git revert -m 1`. Depois dele, voltar ao C exige reverter também a remoção ou partir de `legacy/c-runtime-final`; o plano trata isso como fora do caminho normal. + +**Pré-condições (todas obrigatórias, conferidas no PR):** +- [ ] Task 5.1 concluída (um ciclo de release sem revert). +- [ ] Gate 4.2 com `gate: PASS` sobre o commit-base do PR. +- [ ] D1–D4 registradas e aprovadas. +- [ ] Nenhuma issue aberta com o label `area:memtrack`/`kind:bug` sem triagem. + +### Task 6.1: Atualizar a tag de preservação + +- [ ] **Step 1:** `git tag -s -f legacy/c-runtime-final origin/develop -m "Último commit com o runtime C (antes da remoção)" && git push -f origin legacy/c-runtime-final`, e anotar o hash no PR da Task 6.2. +- [ ] **Step 2:** Ensaiar a recuperação: em um clone descartável, `git checkout legacy/c-runtime-final`, build e `ctest`. Como a tag aponta para o commit antes da remoção (e depois do merge da 5.1), o ensaio também roda `git revert -m 1 ` e confirma que o runtime C volta a compilar e passar no `ctest`. + +**Critério de conclusão:** o rollback foi ensaiado com sucesso e registrado no PR. + +### Task 6.2: Remover o runtime C, do menos ao mais acoplado + +Um PR, um commit por grupo, e **cada commit** mantém o build + `ctest` + `run-corpus.sh` (com `REPS=1`, como verificação rápida) verdes. Desde a Task 3.1 o runtime C está fora do build, então os commits só apagam código morto; a ordem, das folhas que nenhum outro `.c` usa até o núcleo, mantém cada commit revisável e revertível isoladamente. + +- [ ] **Commit 1:** `WitnessGenerationNone.c`, `AnalysisModeNone.c`, `NonDetGeneratorNone.c` (variantes vazias; nenhum outro arquivo depende delas). +- [ ] **Commit 2:** `ContainerRealloc.c`, `ContainerBTree.c`, `BTree.c`, `BTree.h` (containers; só os logs dependem deles, e os logs já não são ligados). +- [ ] **Commit 3:** `TrackBBLog.c`, `NonDetLog.c`. +- [ ] **Commit 4:** `PropertyGenerator.c`, `WitnessGeneration.c` (canal de resultado; Rust coberto pela Task 2.4). +- [ ] **Commit 5:** `AnalysisModeAssert.c`, `AnalysisModeOverflow.c`. +- [ ] **Commit 6:** `AllocationLog.c`, `ListLog.c`, `HeapLog.c`, `AnalysisModeMemtrack.c`, `AnalysisModeMemcleanup.c`. +- [ ] **Commit 7:** `NonDetGeneratorLibFuzzy.c`, `NonDetGeneratorKlee.c`. +- [ ] **Commit 8:** `WasmRuntimeStubs.c`. +- [ ] **Commit 9:** `Map2CheckFunctions.c` (núcleo; todos dependem dele), `header/*.h`, `lib/CMakeLists.txt`, `lib/NOTE.txt`, `CMakeLists.txt`, `NOTES.txt` e o diretório `modules/backend/library/`. +- [ ] **Commit 10:** `tests/unit/backend-library/` (as asserções já estão em Rust, Task 5.2) e a sua entrada em `tests/CMakeLists.txt`. +- [ ] **Commit 11:** remover os resíduos do runtime C no frontend: constantes e comentários de `caller.cpp`/`caller.hpp` que citam os `.bc` C (o ramo de link já foi trocado na Task 3.3), e qualquer `#include` de `modules/backend/library/header`. +- [ ] **Commit 12:** em `modules/backend/library/NOTES.txt` não há mais nada a migrar (o conteúdo foi para `integration-guide.md` na Task 6.4); confirmar com `git grep -n 'backend/library'` que só restam notas históricas. +- [ ] **Verificação final:** `grep -rnE 'backend/library|\.bc"' modules/frontend | grep -v -- '-output.bc\|-result.bc\|-compiled.bc'` não retorna nada. + +**Critério de conclusão:** o PR foi mergeado; o CI do Map2Check passa sem a coluna `c`; a busca acima retorna vazio. + +### Task 6.3: CI e ferramentas de análise + +- [ ] **Step 1:** Conferir que o `ci.yml` já não tem passos do runtime C (removidos na Task 3.4) e atualizar `.cppcheck-suppressions.txt` e `.clang-tidy` se ainda citarem `modules/backend/library`. +- [ ] **Step 2:** Adicionar ao Map2Check uma versão de `scripts/check-no-c-era-artifacts.sh` (copiada deste repo) com o padrão `^modules/backend/library/` e o seu teste. + +**Critério de conclusão:** a guarda falha se alguém reintroduzir o diretório; o CI passa. + +### Task 6.4: Documentação + +- [ ] **Map2Check `README.md`:** a seção de arquitetura passa a dizer que o runtime é `map2check-library` (Rust), com a versão mínima, o link e como fixar uma versão. +- [ ] **Map2Check `CLAUDE.md`, `docs/`, `modules/backend/pass/NOTES.txt`:** trocar referências a `modules/backend/library` pelos fontes Rust. As instruções de "como adicionar um tipo nondet" (`modules/backend/library/NOTES.txt`) migram para `map2check-library/docs/integration-guide.md`. +- [ ] **Este repo `README.md` e `CLAUDE.md`:** declarar que esta biblioteca é **o** runtime do Map2Check e citar `legacy/c-library-final` e `legacy/c-runtime-final` como referências históricas. +- [ ] **Este repo `TODO.md`:** marcar os itens concluídos e mover a seção "Dependente do repositório externo" para "Concluído", com os links dos PRs. +- [ ] **Este repo `docs/plano_map2check_library_rust.md` e `audit/`:** acrescentar uma nota no topo dizendo que o documento é histórico e apontando para este plano. Não reescrever evidência. +- [ ] **CHANGELOGs dos dois repositórios:** uma entrada de remoção, com o hash da tag de preservação. + +**Critério de conclusão:** `grep -rn 'backend/library' README.md CLAUDE.md docs/` no Map2Check só aponta para notas históricas. + +--- + +## Fase 7 — (removida) + +A migração dos passes para a API núcleo, antes prevista aqui, virou a Task 3.2 por causa da D5. Não há camada de compatibilidade a aposentar depois do cutover. + +--- + +## Execution order (resumo) + +0.1 → 0.2 → 0.3 → **1.1 (D1, aprovação)** → 1.2 → 2.1 → 2.2 → 2.3 → 2.4 → 2.5 → (2.6 depois de 4.3) → 2.7 → 2.8 → 2.9 → 3.1 → 3.2 → 3.3 → 3.4 → 4.1 → 4.3 → 4.2 → 5.1 → 5.2 → 6.1 → **6.2 (não-retorno)** → 6.3 → 6.4. From 28881cf05f632cbef82409f69dd0fe7322ef25e9 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 19:05:41 -0400 Subject: [PATCH 02/63] docs: retarget the cutover plan at Map2Check develop (AFL++ instead of LibFuzzer) --- .../2026-10-15-cutover-runtime-c-map2check.md | 53 +++++++++++-------- 1 file changed, 32 insertions(+), 21 deletions(-) diff --git a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md index 80bcebb..614a81f 100644 --- a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md +++ b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md @@ -4,9 +4,9 @@ **Goal:** Substituir o runtime C do Map2Check (`hbgit/Map2Check:modules/backend/library`) pela biblioteca Rust deste repositório (`libmap2check`) e remover o runtime C, sem regressão de veredito nem de desempenho e com rollback possível até um ponto de não-retorno explícito. -**Architecture:** O runtime C é compilado em 23 arquivos `.bc` (LLVM 16), e o frontend (`modules/frontend/caller.cpp:226`) os liga ao programa instrumentado com `llvm-link`, escolhendo o conjunto por modo (memtrack, memcleanup, overflow, assert, none), por gerador (none, KLEE, LibFuzzer), por estrutura de dados (Array ou BTree) e por wasm. A biblioteca Rust passa a ser o único runtime. A biblioteca só exporta a sua API núcleo, sem camada de compatibilidade (D5). Primeiro ela ganha paridade funcional. Depois, numa branch do Map2Check, os passes LLVM passam a emitir diretamente a API núcleo e o frontend passa a ligar `libmap2check.a` (migração direta, sem ABI dupla). A branch é validada contra o C por testes diferenciais e mergeada num único merge commit, revertível. Só depois de um ciclo de release o C é removido. +**Architecture:** O runtime C (`develop` do Map2Check, `f01e8c3e`) são 21 arquivos `.c` compilados para `.bc` (LLVM 16). O frontend (`modules/frontend/caller.cpp:550`, `Caller::linkLLVM`) os liga ao programa instrumentado com `llvm-link`, escolhendo o conjunto por modo (memtrack, memcleanup, overflow, assert, reachability, cover-branches), por gerador (none, KLEE, AFL++), por estrutura de dados (Array ou BTree) e por wasm. A biblioteca Rust passa a ser o único runtime. A biblioteca só exporta a sua API núcleo, sem camada de compatibilidade (D5). Primeiro ela ganha paridade funcional. Depois, numa branch do Map2Check, os passes LLVM passam a emitir diretamente a API núcleo e o frontend passa a ligar `libmap2check.a` (migração direta, sem ABI dupla). A branch é validada contra o C por testes diferenciais e mergeada num único merge commit, revertível. Só depois de um ciclo de release o C é removido. -**Tech Stack:** Rust 1.82 (LLVM 19), cbindgen 0.26, serde_json; Map2Check: C++17, CMake, LLVM/Clang 16, KLEE (z3/stp), LibFuzzer, wabt 1.0.41, benchexec. +**Tech Stack:** Rust 1.82 (LLVM 19), cbindgen 0.26, serde_json; Map2Check (`develop`): C++17, CMake, LLVM/Clang 16, KLEE (z3/stp), AFL++ 4.40c (persistent mode, PCGUARD), wabt 1.0.41, benchexec. **Spec:** a mensagem do usuário que pediu este plano (cutover para Rust apenas), mais as respostas da sessão de 2026-10-15: **escopo = runtime C do Map2Check**; **programas C de teste da ABI deste repositório ficam como testes**; **D5: sem camada de compatibilidade** e **migração direta dos passes, sem tabela de ABI dupla** (decisões do usuário). Requisitos de paridade de referência: [TODO.md](../../../TODO.md), issues #6–#24 e #36–#38, [docs/api-deltas.md](../../api-deltas.md) e [docs/semantic-deltas.md](../../semantic-deltas.md). @@ -16,7 +16,8 @@ - Nenhuma funcionalidade é declarada obsoleta sem uma justificativa registrada em `docs/cutover/decisoes.md`, com evidência e aprovação explícita do mantenedor. - Passos pequenos e reversíveis: até a Fase 3, `develop` do Map2Check não muda de runtime; da Task 5.1 até o ponto de não-retorno (Fase 6), voltar ao C é um `git revert -m 1` do merge. - Regressão de desempenho máxima aceitável: **10% na mediana** do tempo de parede por tarefa (TODO.md, "Medir e controlar a regressão de desempenho"). -- Toolchain do Map2Check: **LLVM/Clang 16** (Dockerfile.dev:45-55, cmake/FindLibFuzzer.cmake:22). Toolchain deste repositório: **Rust 1.82.0 com LLVM 19.1.1** (`rust-toolchain.toml`). +- Alvo no Map2Check: branch **`develop`** (`f01e8c3e` em 2026-10-15; 173 commits à frente de `master`). Números de linha citados neste plano referem-se a `develop`, salvo indicação. +- Toolchain do Map2Check: **LLVM/Clang 16** e **AFL++ 4.40c** (`Dockerfile.dev`). Geradores aceitos pelo CLI: `afl` e `symex` (`modules/frontend/map2check.cpp:929`). Toolchain deste repositório: **Rust 1.82.0 com LLVM 19.1.1** (`rust-toolchain.toml`). - Commits com assinatura verificada; tags de preservação assinadas (`git tag -s`). - Sem atribuição de IA em commits, PRs ou issues ([CLAUDE.md](../../../CLAUDE.md)). - O header `include/map2check.h` é gerado por `build.rs`; nunca editá-lo à mão. @@ -26,9 +27,9 @@ ## Review Focus -1. **Tarefa que gera `VERIFICATION SUCCEEDED`:** hoje, só o caminho KLEE (`--nondet-generator symex`) pode provar TRUE (`modules/frontend/map2check.cpp:270-285`). Se o runtime Rust não rodar sob KLEE, toda tarefa TRUE vira silêncio ou UNKNOWN. Esperado: a mesma contagem de TRUE que o C. O teste está na Task 4.2 (o gate exige paridade por gerador, incluindo `symex`). +1. **Tarefa que gera `VERIFICATION SUCCEEDED`:** no inventário feito em `master`, só o caminho KLEE (`--nondet-generator symex`) prova TRUE (`master:modules/frontend/map2check.cpp:270-285`); a Task 0.2 confirmou 4 SUCCEEDED com `symex` em `develop`. Se o runtime Rust não rodar sob KLEE, toda tarefa TRUE vira silêncio ou UNKNOWN. Esperado: a mesma contagem de TRUE que o C. O teste está na Task 4.2 (o gate exige paridade por gerador, incluindo `symex`). 2. **Duas execuções no mesmo diretório de trabalho:** o runtime C sobrescreve `map2check_property`, mas deixa arquivos antigos de propriedade (`map2check_property_free`, ...) que o frontend lê em ordem fixa (`tools.cpp:114-204`). Esperado: o veredito depende só da execução atual. O teste está na Task 2.4 (`stale_property_files_are_removed_on_init`). -3. **Iterações sucessivas do LibFuzzer no mesmo processo:** o estado de uma entrada contamina a seguinte. Esperado: cada `LLVMFuzzerTestOneInput` parte do estado zerado. O teste está na Task 2.1 (#9). +3. **Iterações sucessivas do AFL++ em persistent mode no mesmo processo:** o estado de uma entrada contamina a seguinte. Esperado: cada iteração (`__AFL_LOOP` ou `LLVMFuzzerTestOneInput` via driver) parte do estado zerado. O teste está na Task 2.1 (#9). 4. **Programa sem violação que chama `exit()` no meio, com `--memcleanup-property`:** o C verifica vazamento no `atexit`. Esperado: FALSE-MEMCLEANUP quando há bloco alcançável não liberado. O teste está na Task 2.2 (`memcleanup_reports_leak_at_exit`). 5. **Arquivo `.bc` Rust ligado com `llvm-link-16`:** bitcode do LLVM 19 não é lido pelo LLVM 16 ("Unknown attribute kind" / "Invalid record"). Esperado: o build falha alto e cedo, não na hora da verificação. O teste está na Task 1.1 (spike) e na Task 3.1 (o CMake falha se o artefato não for legível por `llvm-dis-16`). @@ -36,11 +37,14 @@ ## 0. Achados que corrigem a premissa do pedido +> **Revisão de 2026-10-15 (durante a execução):** o inventário original foi feito sobre o `master` local do Map2Check (`83471fba`). O alvo é `develop` (`f01e8c3e`), que trocou LibFuzzer por **AFL++** (`NonDetGeneratorAFL.c` substitui `NonDetGeneratorLibFuzzy.c`; CLI `afl|symex`), acrescentou `--slice`, o modo `COVER_BRANCHES_MODE` e o símbolo `map2check_binop_srem`, e reduziu o runtime a 21 arquivos `.c` (3 039 linhas). As seções abaixo já refletem `develop`. + - **Neste repositório já não há C legado.** A implementação C/C++ foi removida em `f1cbcfe` (84 arquivos, 8 083 linhas), e a remoção já está em `develop` e em `main`. O último commit com o C é `f4c43b5`. A tag `archive/memory_check` aponta para um commit de 2021 e **não** preserva esse estado. O que resta em C são três programas consumidores da ABI (`test/code*_by_caller_test.c`) e o harness `tests/ffi_harness.c`. Por decisão do usuário, eles **ficam** como testes. -- **O runtime C ainda em uso está no Map2Check**, em `modules/backend/library` (3 893 linhas em 33 arquivos). Ele não é a mesma API desta biblioteca: os nomes divergem (`map2check_binop_add` × `map2check_binop_add_int`, `map2check_assert` × `map2check_is_valid_assert`, `map2check_malloc` × `map2check_map_malloc`, `map2check_non_det_int` × `__VERIFIER_nondet_int`). Dos nomes `map2check_*` que aparecem nos passes, cerca de 30 não existem no header Rust. A lista `audit/evidence/missing_compat.txt` dá 33, mas está inflada: `map2check_pointer_cast` (`OverflowPass.cpp:255`) e `map2check_nondet_cast_` (`NonDetPass.cpp:167`) são só nomes de valores do IR (`Twine` de instruções de cast), não chamadas ao runtime. Recalcular na Task 0.3, porque a evidência é de `Map2Check@f01e8c3` e o `master` local está em `83471fba`. Como não haverá camada de compatibilidade (D5), essa diferença se resolve mudando o que os passes emitem (Task 3.2), não a biblioteca. -- **O runtime é ligado como bitcode, não como `.a`.** `caller.cpp:226-312` faz `llvm-link` de `.bc` no programa instrumentado, e esse `.bc` final vai para `clang -fsanitize=fuzzer` ou para o KLEE. O rustc 1.82 emite bitcode LLVM 19, que o `llvm-link-16` não lê. **Esse é o bloqueio técnico central** (Fase 1). +- **O runtime C ainda em uso está no Map2Check**, em `modules/backend/library` (`develop`: 21 `.c` com 3 039 linhas, 13 headers). Ele não é a mesma API desta biblioteca: os nomes divergem (`map2check_binop_add` × `map2check_binop_add_int`, `map2check_assert` × `map2check_is_valid_assert`, `map2check_malloc` × `map2check_map_malloc`, `map2check_non_det_int` × `__VERIFIER_nondet_int`). Dos nomes `map2check_*` que aparecem nos passes, cerca de 30 não existem no header Rust. A lista `audit/evidence/missing_compat.txt` dá 33, mas está inflada: `map2check_pointer_cast` (`OverflowPass.cpp:255`) e `map2check_nondet_cast_` (`NonDetPass.cpp:167`) são só nomes de valores do IR (`Twine` de instruções de cast), não chamadas ao runtime. Recalcular na Task 0.3, porque a evidência é de `Map2Check@f01e8c3` e o `master` local está em `83471fba`. Como não haverá camada de compatibilidade (D5), essa diferença se resolve mudando o que os passes emitem (Task 3.2), não a biblioteca. +- **O runtime é ligado como bitcode, não como `.a`.** `caller.cpp:550-641` faz `llvm-link` de `.bc` no programa instrumentado, e esse `.bc` final vai para `afl-clang-fast` (`caller.cpp:395-440`, binários `-fuzzed.out` e `-cmplog.out`) ou para o KLEE (`caller.cpp:729`). O rustc 1.82 emite bitcode LLVM 19, que o `llvm-link-16` não lê. **Esse é o bloqueio técnico central** (Fase 1). - **O runtime C é composto por modo na hora do link.** Cada `AnalysisMode*.c` define os mesmos símbolos (`analysis_init`, `map2check_malloc`, ...), e o frontend escolhe qual `.bc` ligar. Uma biblioteca Rust única precisa selecionar o modo em tempo de execução. - **O canal de resultado é por arquivos, não JSON.** O frontend lê `map2check_property`, `map2check_property_{free,deref,assert,overflow,memtrack,memcleanup,unknown}`, `list_log.csv`, `klee_log.csv`, `track_bb_log.st` e `map2check_checked_error` (`tools.hpp:43-54`, `tools.cpp:114-204`, `caller.cpp:44`). +- **Gerador de fuzzing divergente:** a biblioteca Rust implementa o gerador para **LibFuzzer** (`LLVMFuzzerTestOneInput`, `rust/src/nondet/libfuzzer.rs`), mas `develop` usa **AFL++ em persistent mode**: `NonDetGeneratorAFL.c` define o `main`, lê os bytes de `__AFL_FUZZ_TESTCASE_BUF` e cancela entradas com `longjmp` em `nondet_assume`. Há dois caminhos: ligar o harness LibFuzzer da biblioteca ao driver libFuzzer do AFL++ (`afl-clang-fast -fsanitize=fuzzer`) ou portar o loop persistente para Rust. A escolha entra na D1. - **Rust → C via FFI:** o único `extern "C"` importado é `__map2check_main__` (`rust/src/nondet/libfuzzer.rs:44`), o `main` renomeado do programa analisado. Não é código C legado, então **permanece por design**. Se o KLEE for adotado, entram também `klee_make_symbolic` e `klee_assume`, que são intrínsecos do KLEE e também permanecem. A libc chega via `std` (malloc, exit), e isso é aceitável. Não há C próprio chamado pelo Rust a eliminar. ## 1. Mapa de paridade @@ -51,15 +55,15 @@ Status: **(a)** portado · **(b)** parcial · **(c)** não portado · **(d)** ca |---|---|---|---|---|---|---| | 1 | `Map2CheckFunctions.c`: init(isSvComp), passo, assume, `crab_assume`, `assume_loop`, `track_bb`, `__VERIFIER_error`, `target_function`, success/error/destroy/exit | `ffi.rs` (init, success, passo, assume, `__VERIFIER_error`) | b | M | 2.3 (modo), 3.2 (passes) | 2.3, 2.5, 3.2 | | 2 | `AnalysisModeAssert.c` | `analysismode/assert.rs` | b (nomes/assinatura: muda o passe) | P | 3.2 | 3.2 | -| 3 | `AnalysisModeOverflow.c` (add/sub/mul/sdiv int e uint) | `analysismode/overflow.rs` | b (nomes: muda o passe; `sdiv` → `div_int`) | P–M | 3.2 | 3.2 | +| 3 | `AnalysisModeOverflow.c` (add/sub/mul/sdiv/**srem** int e uint) | `analysismode/overflow.rs` | b (nomes: muda o passe; `sdiv` → `div_int`; `srem` sem equivalente → export novo) | P–M | 2.5, 3.2 | 2.5, 3.2 | | 4 | `AnalysisModeMemtrack.c` + `AllocationLog.c` + `ListLog.c` + `HeapLog.c` | `memtrack/`, `analysismode/memory.rs` | b (bugs #11, #36, #37, #38; CSVs) | G | 2.1 | 2.1, 2.4 | | 5 | `AnalysisModeMemcleanup.c` | stub `map2check_set_memcleanup` | c (#10) | M | 2.1 | 2.2 | -| 6 | `AnalysisModeNone.c` | — | c (trivial) | P | 2.3 | 2.3 | +| 6 | `AnalysisModeNone.c` (modos none e `COVER_BRANCHES_MODE`, `caller.cpp:604-609`) | — | c (trivial) | P | 2.3 | 2.3 | | 7 | `ContainerRealloc.c` | `Vec` em `AnalysisState` | a (por design) | P (verificar) | — | 4.2 | | 8 | `ContainerBTree.c` + `BTree.c` (persistência em `*.map2check.bin`; opção `--btree`, marcada "experimental") | — | d? (decisão D3) | P se justificado; M se portado | 4.3 | 2.6 | | 9 | `NonDetLog.c` → `klee_log.csv` | `nondet/mod.rs` (JSON) | b (formato) | M | 2.4 | 2.4 | | 10 | `NonDetGeneratorNone.c` | `__VERIFIER_nondet_*` sem feature | b | P | 2.5, 3.2 | 2.5 | -| 11 | `NonDetGeneratorLibFuzzy.c` | `nondet/libfuzzer.rs` | b (#12, #6) | M | 2.1 | 2.5 | +| 11 | `NonDetGeneratorAFL.c` (AFL++ persistent mode: `main`, buffer de bytes, `longjmp` em `assume`) | `nondet/libfuzzer.rs` (LibFuzzer) | c (outro motor: decidir na D1 entre driver libFuzzer do AFL++ ou port do loop) | M–G | 1.1 (D1) | 2.5 | | 12 | `NonDetGeneratorKlee.c` (`klee_make_symbolic`, `klee_assume`) | — (feature `klee` removida do `Cargo.toml`) | c (#20) | G–GG | **1.1 (bloqueante)** | 2.7 | | 13 | `PropertyGenerator.c` → `map2check_property*` | `output/` (só JSON) | c (#19) | M | 2.3 | 2.4 | | 14 | `TrackBBLog.c` → `track_bb_log.st` | `bbtrack/` | b (formato) | M | 2.4 | 2.4 | @@ -76,7 +80,7 @@ Itens fora deste repositório que **continuam em C++** por decisão de escopo: o | ID | Decisão | Opções | Recomendação | Bloqueia | |---|---|---|---|---| -| D1 | Como o runtime Rust entra no pipeline LLVM 16 | (A) emitir bitcode legível pelo LLVM 16; (B) ligar `libmap2check.a` nativo no passo final do `clang`, sem `llvm-link`; (C) migrar o Map2Check para LLVM 19 | **B** para None/LibFuzzer; para KLEE, só o spike decide (A ou C). B não depende da versão do LLVM, porque o objeto é nativo. | 1.2, 2.7, 3.x | +| D1 | Como o runtime Rust entra no pipeline LLVM 16 (e como o AFL++ dirige as entradas) | (A) emitir bitcode legível pelo LLVM 16; (B) ligar `libmap2check.a` nativo no passo final do `afl-clang-fast`, sem `llvm-link`; (C) migrar o Map2Check para LLVM 19. Para o AFL++: (B1) harness `LLVMFuzzerTestOneInput` da biblioteca + driver libFuzzer do AFL++; (B2) port do loop persistente (`__AFL_LOOP`) para Rust | **B** para None/AFL++, com B1 se o spike mostrar throughput ≥ 90% do C; para KLEE, só o spike decide (A ou C). B não depende da versão do LLVM, porque o objeto é nativo. | 1.2, 2.5, 2.7, 3.x | | D2 | O modo `--wasm` entra no cutover? | portar os stubs para Rust; ou manter fora do escopo | Portar. O CI do Map2Check tem o job `e2e-wasm` ativo (`ci.yml:213`), então não há base para declará-lo obsoleto. | 2.8 | | D3 | A opção `--btree` (disco) | portar; ou remover com justificativa | Medir antes (Task 4.3). Se o RSS do Rust nos casos grandes ficar ≤ o RSS do C com `--btree`, remover e registrar a justificativa. | 2.6 | | D4 | Limite de memória (pico de RSS) | — | ≤ 1,25× o C na mediana. O TODO só fixa o limite de tempo; este valor precisa de aprovação. | 4.2 | @@ -141,7 +145,8 @@ Os critérios são medidos pelo gate da Task 4.2 sobre o corpus `tests/regressio | Sem ABI dupla, não há rollback por flag: depois do merge, voltar ao C exige revert | migração direta | trabalho isolado em `feat/rust-runtime`; merge único com merge commit; gate 4.2 duas vezes antes do merge; revert ensaiado no PR (Task 5.1); a branch é rebaseada semanalmente para que o revert continue limpo | 3.x, 5.1 | | Um passe chama função inexistente na biblioteca: o `clang` final só falha com `undefined reference` na hora da verificação | — | gate `symbol-matrix.sh` + `core_api_coverage` + teste de link ponta a ponta no CI do Map2Check | 0.3, 2.5, 3.2 | | Símbolos duplicados se C e Rust forem ligados juntos | os dois definem `__VERIFIER_*` | o build deixa de compilar/instalar os `.bc` C; o teste da Task 3.1 falha se algum for instalado | 3.1 | -| Dependências externas: wabt 1.0.41 em `/opt`, klee-uclibc compilado com clang-16, solvers z3/stp/btor/yices2 | `Dockerfile.dev`, `caller.cpp:389-420` | gate em container `Dockerfile.dev` + toolchain Rust instalada nele; versões fixadas | 3.3 | +| Dependências externas: wabt 1.0.41 em `/opt`, klee-uclibc compilado com clang-16, AFL++ 4.40c, solvers z3/stp/btor/yices2 | `Dockerfile.dev`; a imagem local de julho não tem AFL++ nem `/usr/bin/time` | gate numa imagem reconstruída do `Dockerfile.dev` de `develop` (`map2check-dev:cutover-f01e8c3e`) + toolchain Rust instalada nela; versões fixadas | 0.2, 3.4 | +| O motor de fuzzing muda de semântica (LibFuzzer × AFL++: rejeição por `longjmp`, ciclo de bytes, `cmplog`) | `NonDetGeneratorAFL.c`, `caller.cpp:418-440` | o spike mede caminhos e throughput nas variantes B1/B2 antes de decidir; gate 4.2 por gerador `afl` | 1.1, 4.2 | | Diferença de alocador/endereços muda contraexemplos textuais | endereços aparecem na saída | o gate compara campos normalizados, não o texto | 4.1 | | Unwinding atravessando a FFI | #8 | `panic = "abort"` no perfil de runtime | 2.1 | @@ -194,7 +199,7 @@ Expected: a tag aparece no remoto e `tag -v` mostra "Good signature". - Create (Map2Check): `tests/cutover/baseline-c.tsv` (gerado) **Interfaces:** -- Produces: `run-corpus.sh `, com as variáveis `GENS` (padrão `"fuzzer symex"`), `REPS` (padrão `5`) e `TIMEOUT` (padrão `45`). Gera um TSV com o cabeçalho exato `task prp gen rep verdict tag line function witness wall_s maxrss_kb`, consumido pela Task 4.1. +- Produces: `run-corpus.sh `, com as variáveis `GENS` (padrão `"afl symex"`), `REPS` (padrão `5`) e `TIMEOUT` (padrão `45`). Gera um TSV com o cabeçalho exato `task prp gen rep verdict tag line function witness wall_s maxrss_kb`, consumido pela Task 4.1. - [ ] **Step 1: Escrever o teste do runner com um `map2check` falso** @@ -244,7 +249,7 @@ set -euo pipefail BIN_DIR="$(cd "$1" && pwd)"; OUT="$2" ROOT="$(cd "$(dirname "$0")/../.." && pwd)" CASES="${CASES:-$ROOT/tests/regression_test/test_cases/sv-benchmarks}" -GENS="${GENS:-fuzzer symex}"; REPS="${REPS:-5}"; TIMEOUT="${TIMEOUT:-45}" +GENS="${GENS:-afl symex}"; REPS="${REPS:-5}"; TIMEOUT="${TIMEOUT:-45}" flags_for() { case "$1" in @@ -459,7 +464,7 @@ git commit -S -m "docs: map Map2Check pass ABI to the core Rust API and gate it - Create (este repo, descartável): `spikes/llvm16/` (não vai para `develop`; fica na branch `spike/llvm16`) **Interfaces:** -- Produces: a decisão D1 registrada em `docs/cutover/decisoes.md`, com a rota escolhida por gerador (`none`, `fuzzer`, `symex`) e os comandos exatos de build/link que a Task 3.1 implementa. +- Produces: a decisão D1 registrada em `docs/cutover/decisoes.md`, com a rota escolhida por gerador (`none`, `afl`, `symex`) e os comandos exatos de build/link que a Task 3.1 implementa. - [ ] **Step 1: Montar o programa de prova** @@ -498,6 +503,12 @@ clang-16 -fsanitize=fuzzer prog.bc target/release/libmap2check.a -ldl -lpthread Expected: liga e roda. Registrar se há colisão de `main`/`LLVMFuzzerTestOneInput` com a instrumentação do LibFuzzer. +Repetir com o AFL++ do `develop` (imagem `map2check-dev:cutover-f01e8c3e`), nas duas variantes da D1: +- **B1:** `afl-clang-fast -fsanitize=fuzzer prog.bc target/release/libmap2check.a -ldl -lpthread -lm -o prog.afl` (driver libFuzzer do AFL++ chamando o `LLVMFuzzerTestOneInput` da biblioteca), seguido de `afl-fuzz -i in -o out -V 30 -- ./prog.afl`. +- **B2:** o `NonDetGeneratorAFL.c` atual compilado com `afl-clang-fast`, como referência de throughput. + +Registrar `execs_per_sec` e `saved_crashes` de `out/default/fuzzer_stats` para as duas, e se B1 encontra o `x == 42`. + - [ ] **Step 4: Rota B sob KLEE (`--load` da biblioteca nativa)** ```bash @@ -1131,7 +1142,7 @@ Escrever um teste por hook do enum (28), mais `nonDetGenerator`/`nonDetLogger` p ```cpp TEST(CallerLink, LinksOnlyTheRustRuntime) { - Caller::LinkInputs in{"h", Map2CheckMode::MEMTRACK_MODE, NonDetGenerator::LibFuzzer, + Caller::LinkInputs in{"h", Map2CheckMode::MEMTRACK_MODE, NonDetGenerator::AFLPlusPlus, DataStructure::Array, /*wasm=*/false, /*witness=*/false}; std::string link = buildLinkCommand(in); EXPECT_EQ(link.find("${MAP2CHECK_PATH}/lib/"), std::string::npos); @@ -1142,14 +1153,14 @@ TEST(CallerLink, EveryModeAndGeneratorLinksRuntime) { for (auto mode : {Map2CheckMode::MEMTRACK_MODE, Map2CheckMode::MEMCLEANUP_MODE, Map2CheckMode::OVERFLOW_MODE, Map2CheckMode::ASSERT_MODE, Map2CheckMode::REACHABILITY_MODE}) - for (auto gen : {NonDetGenerator::LibFuzzer, NonDetGenerator::None}) { + for (auto gen : {NonDetGenerator::AFLPlusPlus, NonDetGenerator::None}) { Caller::LinkInputs in{"h", mode, gen, DataStructure::Array, false, false}; EXPECT_NE(buildFuzzerCommand(in).find("libmap2check.a"), std::string::npos); } } ``` -Conferir os nomes exatos dos enumeradores em `modules/frontend/caller.hpp` antes de escrever o teste. Os nomes acima seguem `caller.cpp:254-280` e devem ser ajustados se diferirem. +Os enumeradores seguem `modules/frontend/caller.hpp:21-45` de `develop`; incluir também `COVER_BRANCHES_MODE` no laço. - [ ] **Step 2–4:** Ver falhar, extrair as funções e implementar, `ctest --output-on-failure` verde. - [ ] **Step 5:** Se D3 = remover, adicionar o aviso de `--btree` com o seu teste. @@ -1199,7 +1210,7 @@ def run(c_rows, r_rows, accepted=""): return subprocess.run([sys.executable, HERE / "compare.py", d / "c.tsv", d / "r.tsv", "--accepted", d / "a.tsv"], capture_output=True, text=True) -ROW = "t.yml\tvalid-memsafety\tfuzzer\t{rep}\t{v}\t{tag}\t7\tmain\t0\t{wall}\t1000\n" +ROW = "t.yml\tvalid-memsafety\tafl\t{rep}\t{v}\t{tag}\t7\tmain\t0\t{wall}\t1000\n" def rows(v="FAILED", tag="FALSE-DEREF", wall=1.0): return [ROW.format(rep=i, v=v, tag=tag, wall=wall) for i in range(1, 6)] @@ -1212,7 +1223,7 @@ def test_verdict_change_blocks(): assert p.returncode == 1 and "verdict" in p.stdout def test_accepted_delta_passes(): - acc = "t.yml\tvalid-memsafety\tfuzzer\ttag\tFALSE-DEREF\tFALSE-FREE\tSD-06\n" + acc = "t.yml\tvalid-memsafety\tafl\ttag\tFALSE-DEREF\tFALSE-FREE\tSD-06\n" assert run(rows(), rows(tag="FALSE-FREE"), acc).returncode == 0 def test_slowdown_over_10_percent_blocks(): @@ -1388,7 +1399,7 @@ Um PR, um commit por grupo, e **cada commit** mantém o build + `ctest` + `run-c - [ ] **Commit 4:** `PropertyGenerator.c`, `WitnessGeneration.c` (canal de resultado; Rust coberto pela Task 2.4). - [ ] **Commit 5:** `AnalysisModeAssert.c`, `AnalysisModeOverflow.c`. - [ ] **Commit 6:** `AllocationLog.c`, `ListLog.c`, `HeapLog.c`, `AnalysisModeMemtrack.c`, `AnalysisModeMemcleanup.c`. -- [ ] **Commit 7:** `NonDetGeneratorLibFuzzy.c`, `NonDetGeneratorKlee.c`. +- [ ] **Commit 7:** `NonDetGeneratorAFL.c`, `NonDetGeneratorKlee.c`. - [ ] **Commit 8:** `WasmRuntimeStubs.c`. - [ ] **Commit 9:** `Map2CheckFunctions.c` (núcleo; todos dependem dele), `header/*.h`, `lib/CMakeLists.txt`, `lib/NOTE.txt`, `CMakeLists.txt`, `NOTES.txt` e o diretório `modules/backend/library/`. - [ ] **Commit 10:** `tests/unit/backend-library/` (as asserções já estão em Rust, Task 5.2) e a sua entrada em `tests/CMakeLists.txt`. From b8ed2dae7c74dcc433486fc4e62af74c81d238aa Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 19:31:18 -0400 Subject: [PATCH 03/63] docs: map Map2Check pass ABI to the core Rust API and gate it with a symbol matrix --- docs/cutover/abi-mapping.tsv | 73 ++++++++++++++++++++++++++++++++++ docs/cutover/symbol-matrix.tsv | 72 +++++++++++++++++++++++++++++++++ scripts/symbol-matrix.sh | 39 ++++++++++++++++++ scripts/test-symbol-matrix.sh | 31 +++++++++++++++ 4 files changed, 215 insertions(+) create mode 100644 docs/cutover/abi-mapping.tsv create mode 100644 docs/cutover/symbol-matrix.tsv create mode 100755 scripts/symbol-matrix.sh create mode 100755 scripts/test-symbol-matrix.sh diff --git a/docs/cutover/abi-mapping.tsv b/docs/cutover/abi-mapping.tsv new file mode 100644 index 0000000..1f9c909 --- /dev/null +++ b/docs/cutover/abi-mapping.tsv @@ -0,0 +1,73 @@ +c_symbol core_symbol owner_task notes +map2check_init map2check_set_mode+map2check_init 2.3 C: (i32 isSvComp) → núcleo: set_mode(proposto)+init(void); o arg int isSvComp é morto: Map2CheckFunctions.c:35 grava sv_comp e nenhum arquivo o lê +map2check_success map2check_success - C: () → núcleo: () (LibraryFunctions.hpp:48) +map2check_track_bb map2check_save_basic_block_log - C: (i32 line,ptr fn) → núcleo: (line:i32,fn); mesma ordem (LibraryFunctions.hpp:54, ffi.rs:111); retorno do núcleo é bbtrack_log_t*, ignorado +map2check_target_function map2check_reach_target 2.3 C: (ptr fn,i32 scope,i32 line) (TargetPass.cpp:89); C runtime grava TARGET_REACHED e aborta (Map2CheckFunctions.c:61); núcleo proposto: (fn,scope,line) sem equivalente no header +map2check_assert map2check_is_valid_assert - C: (i32 cond,i32 line,ptr fn) (AssertPass.cpp:23) → núcleo: (line:i32,fn,expression:i32); ORDEM DIFERENTE: cond passa de primeiro a último +map2check_assume __VERIFIER_assume - C: (i32) (NonDetPass.cpp:94) → núcleo: (i32 condition); núcleo faz exit(0) se 0 +map2check_assume_loop __VERIFIER_assume - C: (i1) (LoopPredAssumePass.cpp:28; o C runtime recebe char e faz cast para int) → núcleo: (i32); o passe deve emitir zext i1→i32 +map2check_crab_assume __VERIFIER_assume - idem; sem getOrInsertFunction: nasce do rename de verifier.assume (NonDetPass.cpp:216), o tipo do arg segue o chamador; C runtime recebe char (Map2CheckFunctions.c:46) +map2check_binop_add map2check_binop_add_int - C: (i64 a,i64 b,i32 width,i32 line,i32 scope,ptr fn) (OperationsFunctions.hpp:74) → núcleo: (a:i32,b:i32,line:u32,scope:u32,fn); SEM arg width e só 32 bits: operandos de outra largura perdem a semântica (lacuna a decidir na 2.5) +map2check_binop_add_uint map2check_binop_add_unsigned - idem map2check_binop_add; operandos u32 no núcleo +map2check_binop_sub map2check_binop_sub_int - idem map2check_binop_add (OperationsFunctions.hpp:80) +map2check_binop_sub_uint map2check_binop_sub_unsigned - idem map2check_binop_add; operandos u32 no núcleo +map2check_binop_mul map2check_binop_mul_int - idem map2check_binop_add (OperationsFunctions.hpp:86) +map2check_binop_mul_uint map2check_binop_mul_unsigned - idem map2check_binop_add; operandos u32 no núcleo +map2check_binop_sdiv map2check_binop_div_int - idem map2check_binop_add (OperationsFunctions.hpp:92); C checa divisor zero e MIN/-1 (AnalysisModeOverflow.c:94) +map2check_binop_srem map2check_binop_rem_int 2.5 sem equivalente núcleo; export novo; C: (i64,i64,i32 width,i32 line,i32 scope,ptr fn) (OperationsFunctions.hpp:95); mesmos modos de falha de sdiv (AnalysisModeOverflow.c:100); só existe na develop +map2check_malloc map2check_map_malloc - C: (ptr,i64 size) (MemoryTrackPass.cpp:789) → núcleo: (ptr,size:i32); size estreita de i64 para i32 +map2check_calloc map2check_map_calloc - C: (ptr,i64 qty,i64 size) → núcleo: (ptr,qty:i32,size:i32); mesma ordem, estreita para i32 +map2check_alloca map2check_map_alloca - C: (ptr name,ptr addr,i64 size,i64 size_primitive,i64 line,i64 scope) (MemoryTrackPass.cpp:802) → núcleo: (name,addr,size:i32,size_primitive:i32,line:i32,scope:i32); mesma ordem, estreita para i32 +map2check_non_static_alloca map2check_map_non_static_alloca - SD-12 (#38): unidade de tamanho; assinatura igual a map2check_alloca (MemoryTrackPass.cpp:808); C multiplica size*size_primitive (AnalysisModeMemtrack.c:370) +map2check_function map2check_map_funct_address - C: (ptr name,ptr addr) (MemoryTrackPass.cpp:814) → núcleo: (var_name,ptr_address); igual +map2check_add_store_pointer map2check_map_store_pointer - C: (ptr var,ptr value,i64 scope,ptr name,i64 line,ptr fn) (MemoryTrackPass.cpp:779) → núcleo: (var_address,value,scope:u32,var_name,line:i32,fn); mesma ordem +map2check_load map2check_check_load - C: (ptr,i64 size) (MemoryTrackPass.cpp:770) → núcleo: (ptr,line:i32,scope:u32,size:i32,fn); o passe passa a emitir linha, escopo e função (DebugInfo) +map2check_check_deref map2check_check_deref - ausente no seed; C: (i64 line,ptr fn) (MemoryTrackPass.cpp:785; C runtime só lê a flag ERROR_DEREF setada por load) → núcleo: (ptr,scope:u32,line:u32,fn) com checagem própria; o passe deve passar ptr e escopo, ou omitir a chamada porque check_load já registra a violação (ffi.rs:894) +map2check_free map2check_check_free+map2check_map_free - C: (ptr name,ptr var_slot,i64 scope,i64 line,ptr fn) (MemoryTrackPass.cpp:818) → núcleo (ambos): (name,ptr,scope:u32,line:u32,fn); SEMÂNTICA A CONFIRMAR: o C desreferencia ptr (AnalysisModeMemtrack.c:341) e o núcleo usa ptr direto como chave (ffi.rs:937, FreeChecker); o passe deve passar o valor já carregado do ponteiro +map2check_free_resolved_address map2check_check_free_resolved_address - SD-06 (#37); C: (ptr,i64 line,ptr fn,i1 nullValid) (MemoryTrackPass.cpp:774) → núcleo: (ptr,line:u32,fn,is_null_valid:i16); i1 passa a i16 (zext) +map2check_posix map2check_map_posix_memalign 2.5 chamado em MemoryTrackPass.cpp:120 para posix_memalign; não é obsoleto; C: (ptr,i64 size) (MemoryTrackPass.cpp:793); o C runtime desreferencia ptr (void**) e registra malloc+store_pointer (AnalysisModeMemtrack.c:328); núcleo proposto: (ptr_slot,size:i32) +map2check_wasm_malloc map2check_wasm_malloc 2.8 C: (i64,i64) (MemoryTrackPass.cpp:825) +map2check_wasm_free map2check_wasm_free 2.8 C: (i64,i64) (MemoryTrackPass.cpp:829) +map2check_wasm_check_access map2check_wasm_check_access 2.8 C: (i64,i64) (MemoryTrackPass.cpp:833) +__map2check_main__ __map2check_main__ - definido pelo programa instrumentado, não pela biblioteca +__VERIFIER_assert __VERIFIER_assert - +__VERIFIER_assume __VERIFIER_assume - +__VERIFIER_error __VERIFIER_error - +__VERIFIER_nondet_char __VERIFIER_nondet_char - só comparado pelo nome (exclusões em TrackBasicBlockPass.cpp:113, OverflowPass.cpp:59); não emitido como chamada +__VERIFIER_nondet_int __VERIFIER_nondet_int - só comparado pelo nome (exclusões em TrackBasicBlockPass.cpp:113, OverflowPass.cpp:59); não emitido como chamada +__VERIFIER_nondet_long __VERIFIER_nondet_long - só comparado pelo nome (exclusões em TrackBasicBlockPass.cpp:113, OverflowPass.cpp:59); não emitido como chamada +__VERIFIER_nondet_pointer __VERIFIER_nondet_pointer - só comparado pelo nome (exclusões em TrackBasicBlockPass.cpp:113, OverflowPass.cpp:59); não emitido como chamada +__VERIFIER_nondet_uint __VERIFIER_nondet_uint - só comparado pelo nome (exclusões em TrackBasicBlockPass.cpp:113, OverflowPass.cpp:59); não emitido como chamada +__VERIFIER_nondet_ushort __VERIFIER_nondet_ushort - só comparado pelo nome (exclusões em TrackBasicBlockPass.cpp:113, OverflowPass.cpp:59); não emitido como chamada +map2check_non_det_int __VERIFIER_nondet_int - tipos de NonDetPass.cpp:155-171 (CALL_IF_HELPER) e NonDetPass.hpp:82-97; C: renomeia a chamada __VERIFIER_nondet_ para map2check_non_det_ (NonDetPass.cpp:186) → núcleo: __VERIFIER_nondet_() sem args +map2check_non_det_unsigned __VERIFIER_nondet_uint - idem primeira linha da família +map2check_non_det_char __VERIFIER_nondet_char - idem primeira linha da família +map2check_non_det_pointer __VERIFIER_nondet_pointer - idem primeira linha da família +map2check_non_det_long __VERIFIER_nondet_long - idem primeira linha da família +map2check_non_det_ushort __VERIFIER_nondet_ushort - idem primeira linha da família +map2check_non_det_short __VERIFIER_nondet_short - idem primeira linha da família +map2check_non_det_ulong __VERIFIER_nondet_ulong - idem primeira linha da família +map2check_non_det_bool __VERIFIER_nondet_bool - idem primeira linha da família +map2check_non_det_uchar __VERIFIER_nondet_uchar - idem primeira linha da família +map2check_non_det_pchar __VERIFIER_nondet_pchar 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_non_det_size_t __VERIFIER_nondet_size_t - idem primeira linha da família +map2check_non_det_loff_t __VERIFIER_nondet_loff_t 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_non_det_sector_t __VERIFIER_nondet_sector_t 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_non_det_uint __VERIFIER_nondet_uint - idem primeira linha da família +map2check_non_det_double __VERIFIER_nondet_double - idem primeira linha da família +map2check_nondet_int map2check_save_nondet_log_int - tipos de NonDetFunctions.hpp:77-92 (NON_DET_FUNCTIONS_HELPER); C: (i32 line,i32 scope,i32 value,ptr fn) (double: valor double; pointer: ptrtoint; char/short/long/etc.: cast para i32) → núcleo: (line:int,scope:int,type_var:enum var_type_t,value:T*,fn); o núcleo recebe o tipo e o valor POR PONTEIRO: o passe deve gravar o valor num slot e passar o endereço +map2check_nondet_unsigned map2check_save_nondet_log_unsigned 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_char map2check_save_nondet_log_char - idem primeira linha da família +map2check_nondet_pointer map2check_save_nondet_log_pointer 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_long map2check_save_nondet_log_long - idem primeira linha da família +map2check_nondet_ushort map2check_save_nondet_log_ushort 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_short map2check_save_nondet_log_short 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_ulong map2check_save_nondet_log_ulong 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_bool map2check_save_nondet_log_bool 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_uchar map2check_save_nondet_log_uchar 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_pchar map2check_save_nondet_log_pchar 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_size_t map2check_save_nondet_log_size_t 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_loff_t map2check_save_nondet_log_loff_t 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_sector_t map2check_save_nondet_log_sector_t 2.5 idem primeira linha da família; sem equivalente núcleo; export novo +map2check_nondet_uint map2check_save_nondet_log_uint - idem primeira linha da família +map2check_nondet_double map2check_save_nondet_log_double - idem primeira linha da família diff --git a/docs/cutover/symbol-matrix.tsv b/docs/cutover/symbol-matrix.tsv new file mode 100644 index 0000000..d4658e6 --- /dev/null +++ b/docs/cutover/symbol-matrix.tsv @@ -0,0 +1,72 @@ +symbol core_symbol in_rust_header +map2check_add_store_pointer map2check_map_store_pointer 1 +map2check_alloca map2check_map_alloca 1 +map2check_assert map2check_is_valid_assert 1 +map2check_assume __VERIFIER_assume 1 +map2check_assume_loop __VERIFIER_assume 1 +map2check_binop_add map2check_binop_add_int 1 +map2check_binop_add_uint map2check_binop_add_unsigned 1 +map2check_binop_mul map2check_binop_mul_int 1 +map2check_binop_mul_uint map2check_binop_mul_unsigned 1 +map2check_binop_sdiv map2check_binop_div_int 1 +map2check_binop_srem map2check_binop_rem_int 0 +map2check_binop_sub map2check_binop_sub_int 1 +map2check_binop_sub_uint map2check_binop_sub_unsigned 1 +map2check_calloc map2check_map_calloc 1 +map2check_check_deref map2check_check_deref 1 +map2check_crab_assume __VERIFIER_assume 1 +map2check_free map2check_check_free+map2check_map_free 1 +map2check_free_resolved_address map2check_check_free_resolved_address 1 +map2check_function map2check_map_funct_address 1 +map2check_init map2check_set_mode+map2check_init 0 +map2check_load map2check_check_load 1 +map2check_malloc map2check_map_malloc 1 +map2check_non_det_bool __VERIFIER_nondet_bool 1 +map2check_nondet_bool map2check_save_nondet_log_bool 0 +map2check_non_det_char __VERIFIER_nondet_char 1 +map2check_nondet_char map2check_save_nondet_log_char 1 +map2check_non_det_double __VERIFIER_nondet_double 1 +map2check_nondet_double map2check_save_nondet_log_double 1 +map2check_non_det_int __VERIFIER_nondet_int 1 +map2check_nondet_int map2check_save_nondet_log_int 1 +map2check_non_det_loff_t __VERIFIER_nondet_loff_t 0 +map2check_nondet_loff_t map2check_save_nondet_log_loff_t 0 +map2check_non_det_long __VERIFIER_nondet_long 1 +map2check_nondet_long map2check_save_nondet_log_long 1 +map2check_non_det_pchar __VERIFIER_nondet_pchar 0 +map2check_nondet_pchar map2check_save_nondet_log_pchar 0 +map2check_non_det_pointer __VERIFIER_nondet_pointer 1 +map2check_nondet_pointer map2check_save_nondet_log_pointer 0 +map2check_non_det_sector_t __VERIFIER_nondet_sector_t 0 +map2check_nondet_sector_t map2check_save_nondet_log_sector_t 0 +map2check_non_det_short __VERIFIER_nondet_short 1 +map2check_nondet_short map2check_save_nondet_log_short 0 +map2check_non_det_size_t __VERIFIER_nondet_size_t 1 +map2check_nondet_size_t map2check_save_nondet_log_size_t 0 +map2check_non_det_uchar __VERIFIER_nondet_uchar 1 +map2check_nondet_uchar map2check_save_nondet_log_uchar 0 +map2check_non_det_uint __VERIFIER_nondet_uint 1 +map2check_nondet_uint map2check_save_nondet_log_uint 1 +map2check_non_det_ulong __VERIFIER_nondet_ulong 1 +map2check_nondet_ulong map2check_save_nondet_log_ulong 0 +map2check_non_det_unsigned __VERIFIER_nondet_uint 1 +map2check_nondet_unsigned map2check_save_nondet_log_unsigned 0 +map2check_non_det_ushort __VERIFIER_nondet_ushort 1 +map2check_nondet_ushort map2check_save_nondet_log_ushort 0 +map2check_non_static_alloca map2check_map_non_static_alloca 1 +map2check_posix map2check_map_posix_memalign 0 +map2check_success map2check_success 1 +map2check_target_function map2check_reach_target 0 +map2check_track_bb map2check_save_basic_block_log 1 +map2check_wasm_check_access map2check_wasm_check_access 0 +map2check_wasm_free map2check_wasm_free 0 +map2check_wasm_malloc map2check_wasm_malloc 0 +__VERIFIER_assert __VERIFIER_assert 1 +__VERIFIER_assume __VERIFIER_assume 1 +__VERIFIER_error __VERIFIER_error 1 +__VERIFIER_nondet_char __VERIFIER_nondet_char 1 +__VERIFIER_nondet_int __VERIFIER_nondet_int 1 +__VERIFIER_nondet_long __VERIFIER_nondet_long 1 +__VERIFIER_nondet_pointer __VERIFIER_nondet_pointer 1 +__VERIFIER_nondet_uint __VERIFIER_nondet_uint 1 +__VERIFIER_nondet_ushort __VERIFIER_nondet_ushort 1 diff --git a/scripts/symbol-matrix.sh b/scripts/symbol-matrix.sh new file mode 100755 index 0000000..4e825d4 --- /dev/null +++ b/scripts/symbol-matrix.sh @@ -0,0 +1,39 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: GPL-3.0-only +# Cruza os símbolos de runtime emitidos pelos passes do Map2Check, traduzidos +# por docs/cutover/abi-mapping.tsv, com os declarados em include/map2check.h. +# Sai com 1 se algum símbolo não tiver mapeamento ou se o alvo não existir. +# Uso: symbol-matrix.sh +set -euo pipefail +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +M2C="$1"; MAP="$2" +PASSES=("$M2C"/modules/backend/pass/*.cpp "$M2C"/modules/backend/pass/*.hpp) +# Nomes de valores do IR (Twine de casts), não chamadas ao runtime. +IR_VALUE_NAMES='^(map2check_pointer_cast|map2check_nondet_cast_.*)$' +literais="$(grep -ohE '"(map2check_|__VERIFIER_|__map2check)[A-Za-z0-9_]*[A-Za-z0-9]"' \ + "${PASSES[@]}" 2>/dev/null | tr -d '"' | grep -vE "$IR_VALUE_NAMES" || true)" +# Famílias por tipo geradas por macro (o nome é concatenado com #c_type e não +# aparece como literal): NON_DET_FUNCTIONS_HELPER(X, t) declara +# map2check_nondet_; CALL_IF_HELPER(X, X, t) casa map2check_non_det_. +nondet="$(grep -ohE 'NON_DET_FUNCTIONS_HELPER(_DOUBLE)?\([A-Z][A-Za-z_]*, *[a-z_]+\)' \ + "${PASSES[@]}" 2>/dev/null | sed -E 's/.*, *([a-z_]+)\)/map2check_nondet_\1/' || true)" +non_det="$(grep -ohE 'CALL_IF_HELPER\([A-Z][A-Za-z_]*, *[A-Z_]+, *[a-z_]+\)' \ + "${PASSES[@]}" 2>/dev/null | sed -E 's/.*, *([a-z_]+)\)/map2check_non_det_\1/' || true)" +pass="$(printf '%s\n%s\n%s\n' "$literais" "$nondet" "$non_det" | grep -v '^$' | sort -u || true)" +hdr="$(grep -oE '\b(map2check_[a-z0-9_]+|__VERIFIER_[a-z0-9_]+)\s*\(' \ + "$ROOT/include/map2check.h" | tr -d '( ' | sort -u)" +missing=0 +printf 'symbol\tcore_symbol\tin_rust_header\n' +while IFS= read -r s; do + [ -z "$s" ] && continue + core="$(awk -F'\t' -v s="$s" 'NR>1 && $1==s {print $2}' "$MAP")" + if [ -z "$core" ]; then printf '%s\t-\t0\n' "$s"; missing=1; continue; fi + ok=1 + if [ "$core" != "-" ] && [ "$core" != "__map2check_main__" ]; then + IFS='+' read -ra parts <<<"$core" + for c in "${parts[@]}"; do grep -qxF "$c" <<<"$hdr" || ok=0; done + fi + [ "$ok" = 1 ] || missing=1 + printf '%s\t%s\t%s\n' "$s" "$core" "$ok" +done <<<"$pass" +exit "$missing" diff --git a/scripts/test-symbol-matrix.sh b/scripts/test-symbol-matrix.sh new file mode 100755 index 0000000..444b1ae --- /dev/null +++ b/scripts/test-symbol-matrix.sh @@ -0,0 +1,31 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: GPL-3.0-only +# Teste offline de scripts/symbol-matrix.sh (Task 0.3 do cutover), com passes falsos. +set -euo pipefail +HERE="$(cd "$(dirname "$0")" && pwd)" +T="$(mktemp -d)"; trap 'rm -rf "$T"' EXIT +mkdir -p "$T/modules/backend/pass" +cat > "$T/modules/backend/pass/P.cpp" <<'CPP' +f("map2check_init"); f("map2check_alloca"); Twine b("map2check_pointer_cast"); f("map2check_unmapped"); +CPP +printf 'c_symbol\tcore_symbol\towner_task\tnotes\nmap2check_init\tmap2check_init\t-\t\nmap2check_alloca\tmap2check_map_alloca\t-\t\n' > "$T/map.tsv" +out="$("$HERE/symbol-matrix.sh" "$T" "$T/map.tsv" || true)" +grep -qP '^map2check_init\tmap2check_init\t1$' <<<"$out" +grep -qP '^map2check_alloca\tmap2check_map_alloca\t1$' <<<"$out" +grep -qP '^map2check_unmapped\t-\t0$' <<<"$out" +! grep -q pointer_cast <<<"$out" +if "$HERE/symbol-matrix.sh" "$T" "$T/map.tsv" >/dev/null; then echo "deveria falhar" >&2; exit 1; fi + +# Famílias geradas por macro (sem literal de string no fonte) também são símbolos emitidos +cat > "$T/modules/backend/pass/N.hpp" <<'CPP' +#define NON_DET_FUNCTIONS_HELPER(type, c_type) x("map2check_nondet_" #c_type) +NON_DET_FUNCTIONS_HELPER(Integer, int) +NON_DET_FUNCTIONS_HELPER_DOUBLE(Double, double) +CALL_IF_HELPER(Unsigned, UNSIGNED, unsigned) +CPP +out="$("$HERE/symbol-matrix.sh" "$T" "$T/map.tsv" || true)" +grep -qP '^map2check_nondet_int\t-\t0$' <<<"$out" +grep -qP '^map2check_nondet_double\t-\t0$' <<<"$out" +grep -qP '^map2check_non_det_unsigned\t-\t0$' <<<"$out" +! grep -qP '^map2check_nondet_c_type|^map2check_non_det_c_type' <<<"$out" +echo ok From 2d13ac2c38d1da62a1b4a16f176a85077ae78722 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 19:43:34 -0400 Subject: [PATCH 04/63] docs: fix nondet log contract in the cutover plan to match the core API --- .../superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md index 614a81f..dc1bea2 100644 --- a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md +++ b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md @@ -854,7 +854,7 @@ Pela D5, **não há camada de compatibilidade**: nenhum nome ou assinatura do ru - Produces: - `extern "C" fn map2check_map_posix_memalign(ptr_address: *const c_void, size: i32)`, com a semântica de `map2check_posix` (`AnalysisModeMemtrack.c:328`). - `__VERIFIER_nondet_{loff_t,sector_t,pchar}` e `map2check_save_nondet_log_` para todo tipo listado no mapeamento (#6), com a representação C de cada tipo confirmada no `NonDetGenerator.h` do Map2Check. - - A decisão do #13 registrada em `docs/api-deltas.md`: a assinatura de `map2check_save_nondet_log_*` continua a núcleo (valor por parâmetro, retorno `void`), e o passe se adapta. + - A decisão do #13 registrada em `docs/api-deltas.md`: a assinatura de `map2check_save_nondet_log_*` continua a núcleo atual de `rust/src/ffi.rs` (`line: i32, scope: i32, type_var: i32, value: *mut T, function_name` → `*mut non_det_log_t`), e o passe se adapta (grava o valor num `alloca` e passa o ponteiro). - Ao final, todo `core_symbol` do mapeamento existe no header. A Task 3.2 depende de `scripts/symbol-matrix.sh` sair com 0. - [ ] **Step 1: Teste que exige cada alvo do mapeamento no artefato** From 2a0a68f61ae365c35ffc4d1432286922e7b4a7c6 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 19:45:15 -0400 Subject: [PATCH 05/63] docs: fix the cutover ABI mapping after review (unsigned binops, widths, overflowError, termination notes) --- docs/cutover/abi-mapping.tsv | 61 +++++++++++++++++----------------- docs/cutover/symbol-matrix.tsv | 19 ++++++----- scripts/symbol-matrix.sh | 7 +++- scripts/test-symbol-matrix.sh | 10 ++++++ 4 files changed, 57 insertions(+), 40 deletions(-) diff --git a/docs/cutover/abi-mapping.tsv b/docs/cutover/abi-mapping.tsv index 1f9c909..a4b324a 100644 --- a/docs/cutover/abi-mapping.tsv +++ b/docs/cutover/abi-mapping.tsv @@ -2,34 +2,35 @@ c_symbol core_symbol owner_task notes map2check_init map2check_set_mode+map2check_init 2.3 C: (i32 isSvComp) → núcleo: set_mode(proposto)+init(void); o arg int isSvComp é morto: Map2CheckFunctions.c:35 grava sv_comp e nenhum arquivo o lê map2check_success map2check_success - C: () → núcleo: () (LibraryFunctions.hpp:48) map2check_track_bb map2check_save_basic_block_log - C: (i32 line,ptr fn) → núcleo: (line:i32,fn); mesma ordem (LibraryFunctions.hpp:54, ffi.rs:111); retorno do núcleo é bbtrack_log_t*, ignorado -map2check_target_function map2check_reach_target 2.3 C: (ptr fn,i32 scope,i32 line) (TargetPass.cpp:89); C runtime grava TARGET_REACHED e aborta (Map2CheckFunctions.c:61); núcleo proposto: (fn,scope,line) sem equivalente no header -map2check_assert map2check_is_valid_assert - C: (i32 cond,i32 line,ptr fn) (AssertPass.cpp:23) → núcleo: (line:i32,fn,expression:i32); ORDEM DIFERENTE: cond passa de primeiro a último +map2check_target_function map2check_reach_target 2.3 C: (ptr fn,i32 scope,i32 line) (TargetPass.cpp:89); C runtime grava TARGET_REACHED e aborta (Map2CheckFunctions.c:61); núcleo proposto: (fn,scope,line) sem equivalente no header; C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (Map2CheckFunctions.c:64) +map2check_assert map2check_is_valid_assert - C: (i32 cond,i32 line,ptr fn) (AssertPass.cpp:23) → núcleo: (line:i32,fn,expression:i32); ORDEM DIFERENTE: cond passa de primeiro a último; C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (map2check_error em AnalysisModeAssert.c:20); is_valid_assert imprime o JSON na hora sem encerrar, então um map2check_success posterior imprime um segundo relatório map2check_assume __VERIFIER_assume - C: (i32) (NonDetPass.cpp:94) → núcleo: (i32 condition); núcleo faz exit(0) se 0 map2check_assume_loop __VERIFIER_assume - C: (i1) (LoopPredAssumePass.cpp:28; o C runtime recebe char e faz cast para int) → núcleo: (i32); o passe deve emitir zext i1→i32 map2check_crab_assume __VERIFIER_assume - idem; sem getOrInsertFunction: nasce do rename de verifier.assume (NonDetPass.cpp:216), o tipo do arg segue o chamador; C runtime recebe char (Map2CheckFunctions.c:46) -map2check_binop_add map2check_binop_add_int - C: (i64 a,i64 b,i32 width,i32 line,i32 scope,ptr fn) (OperationsFunctions.hpp:74) → núcleo: (a:i32,b:i32,line:u32,scope:u32,fn); SEM arg width e só 32 bits: operandos de outra largura perdem a semântica (lacuna a decidir na 2.5) -map2check_binop_add_uint map2check_binop_add_unsigned - idem map2check_binop_add; operandos u32 no núcleo -map2check_binop_sub map2check_binop_sub_int - idem map2check_binop_add (OperationsFunctions.hpp:80) -map2check_binop_sub_uint map2check_binop_sub_unsigned - idem map2check_binop_add; operandos u32 no núcleo -map2check_binop_mul map2check_binop_mul_int - idem map2check_binop_add (OperationsFunctions.hpp:86) -map2check_binop_mul_uint map2check_binop_mul_unsigned - idem map2check_binop_add; operandos u32 no núcleo -map2check_binop_sdiv map2check_binop_div_int - idem map2check_binop_add (OperationsFunctions.hpp:92); C checa divisor zero e MIN/-1 (AnalysisModeOverflow.c:94) -map2check_binop_srem map2check_binop_rem_int 2.5 sem equivalente núcleo; export novo; C: (i64,i64,i32 width,i32 line,i32 scope,ptr fn) (OperationsFunctions.hpp:95); mesmos modos de falha de sdiv (AnalysisModeOverflow.c:100); só existe na develop +map2check_binop_add map2check_binop_add_signed 2.5 C: (i64 a,i64 b,i32 width,i32 line,i32 scope,ptr fn) (OperationsFunctions.hpp:74) → núcleo novo: (a:int64,b:int64,bits:uint32,line:uint32,fn); o núcleo não recebe scope e passa a honrar a largura que o passe já envia; overflow reporta OVERFLOW (AnalysisModeOverflow.c:197-206); C encerra na primeira violação (overflowError, AnalysisModeOverflow.c:185) (map2check_error, Map2CheckFunctions.c:72); núcleo registra e continua; SD-03/#36 (Task 2.1) +map2check_binop_add_uint - - C é no-op deliberado (AnalysisModeOverflow.c:219: unsigned wraparound is not overflow); o passe deixa de emitir (OperationsFunctions.hpp:77) +map2check_binop_sub map2check_binop_sub_signed 2.5 C: (i64 a,i64 b,i32 width,i32 line,i32 scope,ptr fn) (OperationsFunctions.hpp:80) → núcleo novo: (a:int64,b:int64,bits:uint32,line:uint32,fn); o núcleo não recebe scope e passa a honrar a largura que o passe já envia; overflow reporta OVERFLOW (AnalysisModeOverflow.c:197-206); C encerra na primeira violação (overflowError, AnalysisModeOverflow.c:185) (map2check_error, Map2CheckFunctions.c:72); núcleo registra e continua; SD-03/#36 (Task 2.1) +map2check_binop_sub_uint - - C é no-op deliberado (AnalysisModeOverflow.c:219: unsigned wraparound is not overflow); o passe deixa de emitir (OperationsFunctions.hpp:83) +map2check_binop_mul map2check_binop_mul_signed 2.5 C: (i64 a,i64 b,i32 width,i32 line,i32 scope,ptr fn) (OperationsFunctions.hpp:86) → núcleo novo: (a:int64,b:int64,bits:uint32,line:uint32,fn); o núcleo não recebe scope e passa a honrar a largura que o passe já envia; overflow reporta OVERFLOW (AnalysisModeOverflow.c:197-206); C encerra na primeira violação (overflowError, AnalysisModeOverflow.c:185) (map2check_error, Map2CheckFunctions.c:72); núcleo registra e continua; SD-03/#36 (Task 2.1) +map2check_binop_mul_uint - - C é no-op deliberado (AnalysisModeOverflow.c:219: unsigned wraparound is not overflow); o passe deixa de emitir (OperationsFunctions.hpp:89) +map2check_binop_sdiv map2check_binop_div_signed 2.5 C: (i64 a,i64 b,i32 width,i32 line,i32 scope,ptr fn) (OperationsFunctions.hpp:92) → núcleo novo: (a:int64,b:int64,bits:uint32,line:uint32,fn); o núcleo não recebe scope e passa a honrar a largura que o passe já envia; divisão por zero reporta FALSE-DIVBYZERO (novo ViolatedProperty, owner 2.5), não OVERFLOW, como no C (AnalysisModeOverflow.c:252 e :266, divisionByZeroError :191); MIN/-1 reporta OVERFLOW; C encerra na primeira violação (overflowError, AnalysisModeOverflow.c:185) (map2check_error, Map2CheckFunctions.c:72); núcleo registra e continua; SD-03/#36 (Task 2.1) +map2check_binop_srem map2check_binop_rem_signed 2.5 C: (i64 a,i64 b,i32 width,i32 line,i32 scope,ptr fn) (OperationsFunctions.hpp:95) → núcleo novo: (a:int64,b:int64,bits:uint32,line:uint32,fn); o núcleo não recebe scope e passa a honrar a largura que o passe já envia; divisão por zero reporta FALSE-DIVBYZERO (novo ViolatedProperty, owner 2.5), não OVERFLOW, como no C (AnalysisModeOverflow.c:252 e :266, divisionByZeroError :191); MIN/-1 reporta OVERFLOW; C encerra na primeira violação (overflowError, AnalysisModeOverflow.c:185) (map2check_error, Map2CheckFunctions.c:72); núcleo registra e continua; SD-03/#36 (Task 2.1) map2check_malloc map2check_map_malloc - C: (ptr,i64 size) (MemoryTrackPass.cpp:789) → núcleo: (ptr,size:i32); size estreita de i64 para i32 map2check_calloc map2check_map_calloc - C: (ptr,i64 qty,i64 size) → núcleo: (ptr,qty:i32,size:i32); mesma ordem, estreita para i32 map2check_alloca map2check_map_alloca - C: (ptr name,ptr addr,i64 size,i64 size_primitive,i64 line,i64 scope) (MemoryTrackPass.cpp:802) → núcleo: (name,addr,size:i32,size_primitive:i32,line:i32,scope:i32); mesma ordem, estreita para i32 map2check_non_static_alloca map2check_map_non_static_alloca - SD-12 (#38): unidade de tamanho; assinatura igual a map2check_alloca (MemoryTrackPass.cpp:808); C multiplica size*size_primitive (AnalysisModeMemtrack.c:370) map2check_function map2check_map_funct_address - C: (ptr name,ptr addr) (MemoryTrackPass.cpp:814) → núcleo: (var_name,ptr_address); igual map2check_add_store_pointer map2check_map_store_pointer - C: (ptr var,ptr value,i64 scope,ptr name,i64 line,ptr fn) (MemoryTrackPass.cpp:779) → núcleo: (var_address,value,scope:u32,var_name,line:i32,fn); mesma ordem -map2check_load map2check_check_load - C: (ptr,i64 size) (MemoryTrackPass.cpp:770) → núcleo: (ptr,line:i32,scope:u32,size:i32,fn); o passe passa a emitir linha, escopo e função (DebugInfo) -map2check_check_deref map2check_check_deref - ausente no seed; C: (i64 line,ptr fn) (MemoryTrackPass.cpp:785; C runtime só lê a flag ERROR_DEREF setada por load) → núcleo: (ptr,scope:u32,line:u32,fn) com checagem própria; o passe deve passar ptr e escopo, ou omitir a chamada porque check_load já registra a violação (ffi.rs:894) -map2check_free map2check_check_free+map2check_map_free - C: (ptr name,ptr var_slot,i64 scope,i64 line,ptr fn) (MemoryTrackPass.cpp:818) → núcleo (ambos): (name,ptr,scope:u32,line:u32,fn); SEMÂNTICA A CONFIRMAR: o C desreferencia ptr (AnalysisModeMemtrack.c:341) e o núcleo usa ptr direto como chave (ffi.rs:937, FreeChecker); o passe deve passar o valor já carregado do ponteiro -map2check_free_resolved_address map2check_check_free_resolved_address - SD-06 (#37); C: (ptr,i64 line,ptr fn,i1 nullValid) (MemoryTrackPass.cpp:774) → núcleo: (ptr,line:u32,fn,is_null_valid:i16); i1 passa a i16 (zext) +map2check_load map2check_check_load - C: (ptr,i64 size) (MemoryTrackPass.cpp:770) → núcleo: (ptr,line:i32,scope:u32,size:i32,fn); o passe passa a emitir linha, escopo e função (DebugInfo); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (a flag ERROR_DEREF de AnalysisModeMemtrack.c:158 só encerra em check_deref); check_load grava a violação direto +map2check_check_deref - - check_load já registra a violação (ffi.rs:894, LoadChecker); C: (i64 line,ptr fn) (MemoryTrackPass.cpp:785) só lê a flag ERROR_DEREF e encerra (AnalysisModeMemtrack.c:297); o passe deixa de emitir; C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) +map2check_free map2check_check_free+map2check_map_free - C: (ptr name,ptr var_slot,i64 scope,i64 line,ptr fn) (MemoryTrackPass.cpp:818) → núcleo (ambos): (name,ptr,scope:u32,line:u32,fn); semântica confirmada: o núcleo é chaveado no endereço de heap carregado (ffi_tests.rs:722-744), então o passe carrega *ptr antes da chamada e a ordem é check_free depois map_free; o passe não chama nenhum dos dois se o valor carregado for NULL, como o C (AnalysisModeMemtrack.c:342); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:352) +map2check_free_resolved_address map2check_check_free_resolved_address - SD-06 (#37); C: (ptr,i64 line,ptr fn,i1 nullValid) (MemoryTrackPass.cpp:774) → núcleo: (ptr,line:u32,fn,is_null_valid:i16); i1 passa a i16 (zext); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:170) map2check_posix map2check_map_posix_memalign 2.5 chamado em MemoryTrackPass.cpp:120 para posix_memalign; não é obsoleto; C: (ptr,i64 size) (MemoryTrackPass.cpp:793); o C runtime desreferencia ptr (void**) e registra malloc+store_pointer (AnalysisModeMemtrack.c:328); núcleo proposto: (ptr_slot,size:i32) map2check_wasm_malloc map2check_wasm_malloc 2.8 C: (i64,i64) (MemoryTrackPass.cpp:825) map2check_wasm_free map2check_wasm_free 2.8 C: (i64,i64) (MemoryTrackPass.cpp:829) map2check_wasm_check_access map2check_wasm_check_access 2.8 C: (i64,i64) (MemoryTrackPass.cpp:833) __map2check_main__ __map2check_main__ - definido pelo programa instrumentado, não pela biblioteca +overflowError - - morto: errorLines sempre vazio (OverflowPass.cpp:226); C: (i32 line,ptr fn) (OperationsFunctions.hpp:98) reporta FALSE_OVERFLOW e encerra (AnalysisModeOverflow.c:185) __VERIFIER_assert __VERIFIER_assert - __VERIFIER_assume __VERIFIER_assume - __VERIFIER_error __VERIFIER_error - @@ -55,19 +56,19 @@ map2check_non_det_loff_t __VERIFIER_nondet_loff_t 2.5 idem primeira linha da fam map2check_non_det_sector_t __VERIFIER_nondet_sector_t 2.5 idem primeira linha da família; sem equivalente núcleo; export novo map2check_non_det_uint __VERIFIER_nondet_uint - idem primeira linha da família map2check_non_det_double __VERIFIER_nondet_double - idem primeira linha da família -map2check_nondet_int map2check_save_nondet_log_int - tipos de NonDetFunctions.hpp:77-92 (NON_DET_FUNCTIONS_HELPER); C: (i32 line,i32 scope,i32 value,ptr fn) (double: valor double; pointer: ptrtoint; char/short/long/etc.: cast para i32) → núcleo: (line:int,scope:int,type_var:enum var_type_t,value:T*,fn); o núcleo recebe o tipo e o valor POR PONTEIRO: o passe deve gravar o valor num slot e passar o endereço -map2check_nondet_unsigned map2check_save_nondet_log_unsigned 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_char map2check_save_nondet_log_char - idem primeira linha da família -map2check_nondet_pointer map2check_save_nondet_log_pointer 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_long map2check_save_nondet_log_long - idem primeira linha da família -map2check_nondet_ushort map2check_save_nondet_log_ushort 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_short map2check_save_nondet_log_short 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_ulong map2check_save_nondet_log_ulong 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_bool map2check_save_nondet_log_bool 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_uchar map2check_save_nondet_log_uchar 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_pchar map2check_save_nondet_log_pchar 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_size_t map2check_save_nondet_log_size_t 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_loff_t map2check_save_nondet_log_loff_t 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_sector_t map2check_save_nondet_log_sector_t 2.5 idem primeira linha da família; sem equivalente núcleo; export novo -map2check_nondet_uint map2check_save_nondet_log_uint - idem primeira linha da família -map2check_nondet_double map2check_save_nondet_log_double - idem primeira linha da família +map2check_nondet_int map2check_save_nondet_log_int - tipos de NonDetFunctions.hpp:77-92 (NON_DET_FUNCTIONS_HELPER); C: (i32 line,i32 scope,i32 value,ptr fn) (double: valor double; pointer: ptrtoint; char/short/long/etc.: cast para i32) → núcleo (ffi.rs:305): (line:i32,scope:i32,type_var:i32,value:T*,function_name) → non_det_log_t*; o passe grava o valor num alloca e passa o ponteiro + type_var +map2check_nondet_unsigned map2check_save_nondet_log_unsigned 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_char map2check_save_nondet_log_char - idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var +map2check_nondet_pointer map2check_save_nondet_log_pointer 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_long map2check_save_nondet_log_long - idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var +map2check_nondet_ushort map2check_save_nondet_log_ushort 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_short map2check_save_nondet_log_short 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_ulong map2check_save_nondet_log_ulong 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_bool map2check_save_nondet_log_bool 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_uchar map2check_save_nondet_log_uchar 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_pchar map2check_save_nondet_log_pchar 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_size_t map2check_save_nondet_log_size_t 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_loff_t map2check_save_nondet_log_loff_t 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_sector_t map2check_save_nondet_log_sector_t 2.5 idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var; sem equivalente núcleo; export novo +map2check_nondet_uint map2check_save_nondet_log_uint - idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var +map2check_nondet_double map2check_save_nondet_log_double - idem primeira linha da família; o passe grava o valor num alloca e passa o ponteiro + type_var diff --git a/docs/cutover/symbol-matrix.tsv b/docs/cutover/symbol-matrix.tsv index d4658e6..e0633c4 100644 --- a/docs/cutover/symbol-matrix.tsv +++ b/docs/cutover/symbol-matrix.tsv @@ -4,16 +4,16 @@ map2check_alloca map2check_map_alloca 1 map2check_assert map2check_is_valid_assert 1 map2check_assume __VERIFIER_assume 1 map2check_assume_loop __VERIFIER_assume 1 -map2check_binop_add map2check_binop_add_int 1 -map2check_binop_add_uint map2check_binop_add_unsigned 1 -map2check_binop_mul map2check_binop_mul_int 1 -map2check_binop_mul_uint map2check_binop_mul_unsigned 1 -map2check_binop_sdiv map2check_binop_div_int 1 -map2check_binop_srem map2check_binop_rem_int 0 -map2check_binop_sub map2check_binop_sub_int 1 -map2check_binop_sub_uint map2check_binop_sub_unsigned 1 +map2check_binop_add map2check_binop_add_signed 0 +map2check_binop_add_uint - 1 +map2check_binop_mul map2check_binop_mul_signed 0 +map2check_binop_mul_uint - 1 +map2check_binop_sdiv map2check_binop_div_signed 0 +map2check_binop_srem map2check_binop_rem_signed 0 +map2check_binop_sub map2check_binop_sub_signed 0 +map2check_binop_sub_uint - 1 map2check_calloc map2check_map_calloc 1 -map2check_check_deref map2check_check_deref 1 +map2check_check_deref - 1 map2check_crab_assume __VERIFIER_assume 1 map2check_free map2check_check_free+map2check_map_free 1 map2check_free_resolved_address map2check_check_free_resolved_address 1 @@ -61,6 +61,7 @@ map2check_track_bb map2check_save_basic_block_log 1 map2check_wasm_check_access map2check_wasm_check_access 0 map2check_wasm_free map2check_wasm_free 0 map2check_wasm_malloc map2check_wasm_malloc 0 +overflowError - 1 __VERIFIER_assert __VERIFIER_assert 1 __VERIFIER_assume __VERIFIER_assume 1 __VERIFIER_error __VERIFIER_error 1 diff --git a/scripts/symbol-matrix.sh b/scripts/symbol-matrix.sh index 4e825d4..2e3be27 100755 --- a/scripts/symbol-matrix.sh +++ b/scripts/symbol-matrix.sh @@ -19,7 +19,12 @@ nondet="$(grep -ohE 'NON_DET_FUNCTIONS_HELPER(_DOUBLE)?\([A-Z][A-Za-z_]*, *[a-z_ "${PASSES[@]}" 2>/dev/null | sed -E 's/.*, *([a-z_]+)\)/map2check_nondet_\1/' || true)" non_det="$(grep -ohE 'CALL_IF_HELPER\([A-Z][A-Za-z_]*, *[A-Z_]+, *[a-z_]+\)' \ "${PASSES[@]}" 2>/dev/null | sed -E 's/.*, *([a-z_]+)\)/map2check_non_det_\1/' || true)" -pass="$(printf '%s\n%s\n%s\n' "$literais" "$nondet" "$non_det" | grep -v '^$' | sort -u || true)" +# Primeiro argumento literal de getOrInsertFunction, qualquer prefixo (a chamada +# pode quebrar linha); mantém os padrões por prefixo acima. +callees="$(cat "${PASSES[@]}" 2>/dev/null | tr '\n' ' ' \ + | grep -oE 'getOrInsertFunction\( *"[A-Za-z0-9_.]+"' | sed -E 's/.*"([^"]+)"/\1/' \ + | grep -vE "$IR_VALUE_NAMES" || true)" +pass="$(printf '%s\n%s\n%s\n%s\n' "$literais" "$nondet" "$non_det" "$callees" | grep -v '^$' | sort -u || true)" hdr="$(grep -oE '\b(map2check_[a-z0-9_]+|__VERIFIER_[a-z0-9_]+)\s*\(' \ "$ROOT/include/map2check.h" | tr -d '( ' | sort -u)" missing=0 diff --git a/scripts/test-symbol-matrix.sh b/scripts/test-symbol-matrix.sh index 444b1ae..ec0a087 100755 --- a/scripts/test-symbol-matrix.sh +++ b/scripts/test-symbol-matrix.sh @@ -28,4 +28,14 @@ grep -qP '^map2check_nondet_int\t-\t0$' <<<"$out" grep -qP '^map2check_nondet_double\t-\t0$' <<<"$out" grep -qP '^map2check_non_det_unsigned\t-\t0$' <<<"$out" ! grep -qP '^map2check_nondet_c_type|^map2check_non_det_c_type' <<<"$out" + +# Callees sem prefixo conhecido, declarados por getOrInsertFunction (inclusive em várias linhas) +cat > "$T/modules/backend/pass/G.hpp" <<'CPP' +x = M->getOrInsertFunction("overflowError", VoidTy, I32, PtrTy); +y = M->getOrInsertFunction( + "otherCallee", VoidTy); +CPP +out="$("$HERE/symbol-matrix.sh" "$T" "$T/map.tsv" || true)" +grep -qP '^overflowError\t-\t0$' <<<"$out" +grep -qP '^otherCallee\t-\t0$' <<<"$out" echo ok From 62c9b6312346877d8665133c0317233ebd60953e Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 20:09:13 -0400 Subject: [PATCH 06/63] docs: record LLVM 16 linkage spike and propose runtime integration route (D1) --- docs/cutover/decisoes.md | 59 +++++++ docs/cutover/spike-llvm16.md | 334 +++++++++++++++++++++++++++++++++++ 2 files changed, 393 insertions(+) create mode 100644 docs/cutover/decisoes.md create mode 100644 docs/cutover/spike-llvm16.md diff --git a/docs/cutover/decisoes.md b/docs/cutover/decisoes.md new file mode 100644 index 0000000..e158b89 --- /dev/null +++ b/docs/cutover/decisoes.md @@ -0,0 +1,59 @@ +# Decisões do cutover do runtime C do Map2Check + +Registro das decisões D1–D5 do plano `docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md`. Uma decisão só vale depois de aprovada pelo mantenedor (comentário na issue #20). + +## D1 + +**Pergunta:** como o runtime Rust entra no pipeline LLVM 16 do Map2Check (`develop`: clang/LLVM 16.0.6, AFL++ 4.40c, KLEE 3.1), e como o AFL++ dirige as entradas. + +**Status: PROPOSTA — aguardando aprovação do mantenedor** + +Evidência: `docs/cutover/spike-llvm16.md` (Task 1.1). Código na branch local `spike/llvm16`. + +### Proposta por gerador + +| Gerador | Rota proposta | Evidência principal | +|---|---|---| +| `none` | **B**: `libmap2check.a` nativo (rustc do repo, 1.82) ligado no passo final, sem `llvm-link` | A rota B liga e roda com clang-16 (`rc=0`). O frontend de `develop` não executa o modo `none` (`Caller::executeAnalysis`: "This mode is not supported"; `map2check.cpp:977` troca `None` por AFL++). Basta que a ligação funcione, e ela usa o mesmo artefato do `afl`. | +| `afl` | **B + B1**: `libmap2check.a` nativo + driver libFuzzer do AFL++ (`afl-clang-fast -fsanitize=fuzzer`), chamando o `LLVMFuzzerTestOneInput` da biblioteca | Em 30 s, o B1 fez 97 474 exec/s contra 42 919 do `NonDetGeneratorAFL.c` (227%; 229% com CmpLog). Os dois acharam `x == 42`, com 1 `saved_crashes` cada. O critério da D1 (≥ 90% do C) é atendido sem portar o `__AFL_LOOP` para Rust (B2). | +| `symex` | **A′**: runtime compilado para bitcode por um rustc com LLVM 16 (1.72.1, LLVM 16.0.5), LTO fat, `opt-16 --strip-debug` só no runtime e `llvm-link-16` com o `-result.bc` | A rota A falha (`llvm-dis-16: error: Unknown attribute kind (91) (Producer: 'LLVM19.1.1-rust-1.82.0-stable' Reader: 'LLVM 16.0.6')`). A rota B sob KLEE dá 1 caminho, sem explorar `x == 42`. A A′ dá 2 caminhos (ktests 42 e 0) na crate `no_std` do brief. A **biblioteca real** (`std`, `serde_json`, `Mutex`) também roda no KLEE 3.1 com as flags do `caller.cpp`: 22 caminhos completos, `x == 42` encontrado. | + +A rota C (Map2Check em LLVM 19) fica como **alternativa futura para `symex`**, não como caminho do cutover. O KLEE 3.3, com "full support for LLVM 17-19", saiu em 2026-10-15 e não foi testado. Migrar o pipeline inteiro custa G (1 a 2 semanas) mais a revalidação dos baselines. + +O `symex` **não está bloqueado**: existe uma rota medida que funciona. A consequência prevista no brief para o caso de nenhuma rota servir não se aplica. O texto era: "o cutover completo fica bloqueado: pela restrição 'não remover sem equivalente validado', o runtime C do KLEE não pode sair". + +### Comandos de build e ligação que a Task 3.1 implementa + +`none` e `afl` (o passe já renomeia `main` para `__map2check_main__`): + +```bash +# biblioteca (rustc do repo) +cargo build --release --features libfuzzer # -> target/release/libmap2check.a +# programa: o -result.bc sai do llvm-link SEM os .bc do runtime C +afl-clang-fast -g -O2 -fsanitize=fuzzer -result.bc libmap2check.a -ldl -lpthread -lm -o -fuzzed.out +AFL_LLVM_CMPLOG=1 afl-clang-fast -g -O2 -fsanitize=fuzzer -result.bc libmap2check.a -ldl -lpthread -lm -o -cmplog.out +afl-fuzz -i afl-in -o afl-out -c ./-cmplog.out -- ./-fuzzed.out +# replay: o crash vai como ARGUMENTO, não pela stdin (ver condição 2) +./-witness-fuzzed.out +``` + +`symex`: + +```bash +# runtime em bitcode LLVM 16 (toolchain separado, LTO fat no perfil) +RUSTUP_TOOLCHAIN=1.72.1 cargo rustc --release --features klee --lib --crate-type staticlib -- --emit=llvm-bc -C panic=abort +opt-16 --strip-debug target/release/deps/map2check-*.bc -o map2check-klee.bc +llvm-link-16 -output.bc map2check-klee.bc -o -witness-result.bc +klee --external-calls=all --optimize --use-cex-cache --libc=uclibc ./-witness-result.bc +``` + +### Condições da proposta (o que a aprovação aceita junto) + +1. **`main` renomeado é obrigatório.** Com o `prog.c` cru, a ligação falha (`multiple definition of 'main'` no LibFuzzer; `undefined reference to '__map2check_main__'` no LibFuzzer e no AFL++). O passe atual já renomeia o `main`; isso não pode regredir. +2. **Replay pela stdin não funciona com B1.** `./b1.afl < crash` não termina (`rc=124` após 5 s), enquanto `./b1.afl crash` aborta (`rc=134`). O frontend (`caller.cpp:957`) hoje redireciona o arquivo para a stdin e precisa passar a usar argumento. +3. **`assume` em persistent mode.** O C faz `longjmp` e segue para a próxima entrada. O Rust faz `exit(0)` (`libfuzzer.rs`, `__VERIFIER_assume`), que derruba o processo persistente. O throughput medido não cobre programas com `assume`. Medir na Task 2.5 antes de fechar o B1. Se cair abaixo de 90% do C, a resposta é B2 (port do loop). +4. **MSRV 1.72 para o build `symex`.** Com rustc 1.72.1, a biblioteca real tem 6 erros de fonte: `unsafe extern` (1), `LazyLock` (4) e `Mutex::clear_poison` (1). Também precisa de um `Cargo.lock` v3 resolvido por MSRV (`serde_json 1.0.154`, `thiserror 2.0.20`), sem `mockall`/`cbindgen` no grafo desse build. Trocas usadas no spike: `extern "C"`, `OnceLock`, sem `clear_poison`. Alternativa: um crate `no_std` separado só para KLEE (rota A′ do brief, comprovada), ao custo de duplicar a lógica de estado. +5. **LTO fat e `--strip-debug` no runtime `symex` são obrigatórios.** Sem LTO: `KLEE: ERROR: Unable to load symbol(__rust_no_alloc_shim_is_unstable)`. Com o debug info do `std`: `LLVM ERROR: Broken module found` ("inlinable function call in a function with debug info must have a !dbg location"). O `-g` do programa analisado é preservado. +6. **Concretizar antes de serializar no `symex`.** O `serde_json` formatando valores simbólicos levou `prog.c` de 2 para 22 caminhos. O gerador KLEE do Rust precisa concretizar (`klee_get_value_*`) o que vai para o log, como o C faz ao escrever por chamada externa. +7. **A rota B sob KLEE não é usada.** Além de não explorar símbolos, o KLEE 3.1 não tem `--load` (`Unknown command line argument '--load=...'`), e a `.so` só carrega com `LD_PRELOAD` e `-Wl,-z,lazy`. +8. **O `rustc 1.72.1` (2023) fica preso ao LLVM 16.** Quando o Map2Check migrar de LLVM, o toolchain do `symex` precisa acompanhar. A rota C remove essa dependência. diff --git a/docs/cutover/spike-llvm16.md b/docs/cutover/spike-llvm16.md new file mode 100644 index 0000000..a4ec7e3 --- /dev/null +++ b/docs/cutover/spike-llvm16.md @@ -0,0 +1,334 @@ +# Spike LLVM 16: como o runtime Rust entra no pipeline do Map2Check (Task 1.1) + +Data: 2026-10-15. Alvo: Map2Check `develop` (worktree `Map2Check-cutover`, HEAD `33b41e81`). +Código do spike: branch local `spike/llvm16` (commit `d13045b`), diretório `spikes/llvm16/`. Ela não vai para `develop`. + +## Ambiente + +| Item | Versão observada | +|---|---| +| Imagem base | `map2check-dev:cutover-f01e8c3e` (Ubuntu jammy, usuário `map2check`, uid 1000) | +| clang / llvm-link / llvm-dis / opt | 16.0.6 (`clang-16`, `llvm-link-16`, `llvm-dis-16`, `opt-16`) | +| KLEE | 3.1 (`/opt/klee/bin/klee`, build Release+Asserts, rev `fe22b907`) | +| AFL++ | 4.40c (`/usr/local/bin/afl-fuzz`, `afl-clang-fast`) | +| rustc do repo | 1.82.0, **LLVM 19.1.1** | +| rustc da Rota A′ | 1.72.1, **LLVM 16.0.5** | + +Imagem derivada `map2check-dev:cutover-rust` (`spikes/llvm16/Dockerfile.rust`): + +```dockerfile +FROM map2check-dev:cutover-f01e8c3e +USER root +ENV RUSTUP_HOME=/opt/rustup CARGO_HOME=/opt/cargo PATH=/opt/cargo/bin:$PATH +RUN curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain 1.82.0 --profile minimal \ + && rustup toolchain install 1.72.1 --profile minimal \ + && chmod -R a+rwX /opt/rustup /opt/cargo +USER map2check +``` + +Todos os comandos rodaram assim (o `release-afl/lib` do Map2Check entra só para leitura): + +```bash +docker run --rm --cpus 4 -v $PWD:/src -w /src \ + -v /home/rock/Documents/Dev/Map2Check-cutover/release-afl/lib:/mc:ro \ + -e CARGO_TARGET_DIR=/src/spikes/llvm16/target-docker -e RUSTUP_TOOLCHAIN=1.82.0 \ + map2check-dev:cutover-rust bash -c '' +``` + +Programas de prova (`spikes/llvm16/`): + +- `prog.c`: o programa do brief, sem alteração. +- `prog42.c`: igual ao `prog.c`, mas com `abort()` em `x == 42`. Com `return 1` o fuzzer não registra nada, porque retorno não é crash. Para contar `saved_crashes` era preciso um sinal. +- `prog42_c.c`: o `prog42.c` como os passes do Map2Check o entregam ao runtime C (`main` → `__map2check_main__`, `__VERIFIER_nondet_int` → `map2check_non_det_int`, `map2check_init(0)`). É a referência B2. + +## Resumo + +| Rota | Esperado (brief) | Observado | Resultado | +|---|---|---|---| +| A: bitcode do rustc 1.82 no `llvm-link-16` | `dis16≠0` | `dis16=1`, `Unknown attribute kind (91)` | **falha (como esperado)** | +| B: `libmap2check.a` nativo + `clang-16 -fsanitize=fuzzer` | liga e roda | com o `prog.c` cru, **não liga** (`main` duplicado + `__map2check_main__` indefinido); com `main` renomeado, liga e roda (`rc=0`) | **passa, desde que o passe renomeie `main`** | +| B1: staticlib + driver libFuzzer do AFL++ | throughput ≥ 90% do C | 97 474 exec/s, 1 crash (`x == 42`) em 0,39 s | **passa** (227% do B2) | +| B2: `NonDetGeneratorAFL.c` (referência) | — | 42 919 exec/s, 1 crash (`x == 42`) em 0,33 s | referência | +| B sob KLEE (`.so` nativo) | 1 caminho, não explora `x == 42` | 1 caminho completo; `x == 42` não explorado | **falha para `symex` (como esperado)** | +| A′: `no_std` + rustc 1.72.1 (LLVM 16) | 2 caminhos | 2 caminhos, ktests `42` e `0` | **passa** | +| A′ estendida: a biblioteca inteira (`std`, `serde_json`) com rustc 1.72.1 | — (fora do brief) | após 6 ajustes de fonte e `Cargo.lock` com resolução por MSRV, roda no KLEE com as flags do Map2Check: 22 caminhos, `x == 42` encontrado | **passa, com ressalvas** | +| C: Map2Check em LLVM 19 | sim/não + estimativa | KLEE 3.3 (2026-10-15) tem suporte completo a LLVM 17–19; passes usam 49 APIs `llvm::` | **viável, não medido** | + +## Rota A: bitcode do rustc 1.82 no LLVM 16 + +```bash +cargo rustc --release --lib --crate-type staticlib -- --emit=llvm-bc -C panic=abort +llvm-dis-16 $CARGO_TARGET_DIR/release/deps/map2check-*.bc -o /dev/null; echo "dis16=$?" +clang-16 -emit-llvm -c spikes/llvm16/prog.c -o prog.bc +llvm-link-16 prog.bc $CARGO_TARGET_DIR/release/deps/map2check-*.bc -o linkedA.bc; echo "link16=$?" +``` + +Saída: + +``` +rustc 1.82.0 (f6e511eec 2024-10-15) LLVM version: 19.1.1 +llvm-dis-16: error: Unknown attribute kind (91) (Producer: 'LLVM19.1.1-rust-1.82.0-stable' Reader: 'LLVM 16.0.6') +dis16=1 +llvm-link-16: target-docker/release/deps/map2check-77edb1d032f8b0d7.bc: error: Unknown attribute kind (91) (Producer: 'LLVM19.1.1-rust-1.82.0-stable' Reader: 'LLVM 16.0.6') +llvm-link-16: error: loading file 'target-docker/release/deps/map2check-77edb1d032f8b0d7.bc' +link16=1 +``` + +Resultado: **falha, como o brief previa.** O bitcode do LLVM 19 não é legível pelo LLVM 16. O formato não tem compatibilidade para trás. + +## Rota B: staticlib nativo no passo final (LibFuzzer) + +```bash +cargo build --release --features libfuzzer +clang-16 -emit-llvm -c prog.c -o prog.bc +clang-16 -fsanitize=fuzzer prog.bc $L -ldl -lpthread -lm -o prog.fuzz # L=.../libmap2check.a +``` + +Com o `prog.c` do brief (o `main` original): + +``` +/usr/bin/ld: /tmp/prog-a25d77.o: in function `main': +prog.c:(.text.main[main]+0x0): multiple definition of `main'; /usr/lib/llvm-16/lib/clang/16/lib/linux/libclang_rt.fuzzer-x86_64.a(FuzzerMain.cpp.o):(.text.main+0x0): first defined here +/usr/bin/ld: .../libmap2check.a(map2check-...rcgu.o): in function `LLVMFuzzerTestOneInput': +map2check...:(.text.LLVMFuzzerTestOneInput+0x49): undefined reference to `__map2check_main__' +clang: error: linker command failed with exit code 1 +``` + +**Há colisão de `main`.** O `LLVMFuzzerTestOneInput` da biblioteca chama `__map2check_main__`, então o programa precisa chegar com o `main` já renomeado, que é o que os passes do Map2Check fazem. Simulando o passe com `-Dmain=__map2check_main__`: + +```bash +clang-16 -emit-llvm -c -Dmain=__map2check_main__ prog.c -o prog_m.bc +clang-16 -fsanitize=fuzzer prog_m.bc $L -ldl -lpthread -lm -o prog_m.fuzz # linkB=0 +./prog_m.fuzz -runs=1000 # rc=0 +``` + +``` +INFO: Loaded 1 modules (3 inline 8-bit counters) +#1000 DONE cov: 2 ft: 3 corp: 1/1b lim: 11 exec/s: 0 rss: 28Mb +Done 1000 runs in 0 second(s) +``` + +Cada execução imprime o relatório JSON (`"result": "TRUE"`) na stdout. Uma rodada de 2 000 000 execuções com `-use_value_profile=1` levou 7 s (285 714 exec/s). A instrumentação SanitizerCoverage entra no bitcode do usuário no passo final (3 contadores), não na biblioteca. + +Resultado: **passa**, com a condição de que o `main` do programa seja renomeado pelo passe antes da ligação. + +## Rotas B1 e B2: AFL++ 4.40c + +### B1: staticlib + driver libFuzzer do AFL++ + +Comando do brief, com o `prog.bc` cru: + +``` +[+] Found '-fsanitize=fuzzer', replacing with libAFLDriver.a +/usr/bin/ld: .../libmap2check.a(...): in function `LLVMFuzzerTestOneInput': +(.text.LLVMFuzzerTestOneInput+0x49): undefined reference to `__map2check_main__' +clang: error: linker command failed with exit code 1 +``` + +Aqui o ligador não acusou `main` duplicado; o único erro é o `__map2check_main__` indefinido. Mesmo assim o `main` precisa ser renomeado. Com o `main` renomeado (`spikes/llvm16/b1build.sh`): + +```bash +clang-16 -emit-llvm -c -Dmain=__map2check_main__ prog42.c -o prog42_m.bc +afl-clang-fast -g -O2 -fsanitize=fuzzer prog42_m.bc $L -ldl -lpthread -lm -o b1.afl +AFL_LLVM_CMPLOG=1 afl-clang-fast -g -O2 -fsanitize=fuzzer prog42_m.bc $L -ldl -lpthread -lm -o b1.cmplog.afl +``` + +Liga. O binário tem `main` do driver, `LLVMFuzzerTestOneInput` da biblioteca, `__map2check_main__` e `__afl_persistent_loop`. Ele também contém o marcador `##SIG_AFL_PERSISTENT##`, então roda em persistent mode. Tamanho: 5 796 288 bytes (B2: 146 512). + +### B2: referência C (`NonDetGeneratorAFL.c`) + +Ligação idêntica à do frontend (`caller.cpp:558–641`, modo reachability, `ContainerRealloc`), com os `.bc` de `release-afl/lib` montados somente leitura (`spikes/llvm16/b2build.sh`): + +```bash +clang-16 -emit-llvm -c prog42_c.c -o prog42_c.bc +llvm-link-16 prog42_c.bc /mc/Map2CheckFunctions.bc /mc/TrackBBLog.bc /mc/NonDetLog.bc \ + /mc/PropertyGenerator.bc /mc/ContainerRealloc.bc /mc/AnalysisModeNone.bc \ + /mc/NonDetGeneratorAFL.bc /mc/WitnessGenerationNone.bc -o result.bc +afl-clang-fast -g -O2 result.bc -o b2.afl +AFL_LLVM_CMPLOG=1 afl-clang-fast -g -O2 result.bc -o b2.cmplog.afl +``` + +### Campanhas de 30 s + +`spikes/llvm16/fuzz.sh`, com o mesmo ambiente que o frontend usa (`caller.cpp:919`), semente `A`, B1 e B2 em paralelo (1 núcleo cada): + +```bash +AFL_NO_UI=1 AFL_NO_AFFINITY=1 AFL_SKIP_CPUFREQ=1 AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES=1 \ +AFL_CRASHING_SEEDS_AS_NEW_CRASH=1 afl-fuzz -i in -o out -V 30 [-c ./X.cmplog.afl] -- ./X.afl +``` + +Valores de `out/default/fuzzer_stats`: + +| Variante | `execs_per_sec` | `execs_done` | `saved_crashes` | `stability` | 1º crash (`time`, `execs`) | `x == 42`? | +|---|---|---|---|---|---|---| +| B1 sem CmpLog | **97 473,82** | 2 924 312 | 1 | 100,00% | 392 ms, 27 994 | sim, entrada `2a 00 00 00 …` | +| B2 sem CmpLog | **42 919,21** | 1 287 791 | 1 | 75,68% | 329 ms, 10 543 | sim, entrada `2a 00 00 00 64 00 18` | +| B1 com CmpLog (`-c`) | 97 270,49 | 2 918 212 | 1 | 100,00% | 3 037 ms, 280 900 | sim | +| B2 com CmpLog (`-c`) | 42 562,56 | 1 276 962 | 1 | 78,38% | 252 ms, 2 748 (`op:its`) | sim, entrada `2a 00 00 00` | + +Throughput B1/B2: **227%** sem CmpLog e **229%** com CmpLog, acima do critério de 90% da D1. Ressalva: o B2 faz E/S de arquivo por execução (`write_property` e `generate_aux_files` em `map2check_success`). O B1 só imprime JSON na stdout, que o afl-fuzz descarta. Parte da diferença é custo de E/S, não do gerador. A comparação justa de throughput fica para quando a saída em arquivo do Rust existir (Task 2.4). + +Em ambos os casos o valor é lido como 4 bytes little-endian do início da entrada, então a semântica do vetor coincide nos dois runtimes. + +### Replay standalone do crash (achado relevante para a Task 3.x) + +O frontend reproduz crashes redirecionando o arquivo para a stdin (`caller.cpp:957`: "the crash file is redirected in"). + +``` +B2: ./b2.afl < crash -> rc=134 (abort) OK +B1: ./b1.afl crash -> rc=134 (abort) OK +B1: ./b1.afl < crash -> rc=124 (timeout 5 s, não termina) +``` + +Pela stdin, o B1 imprime um relatório `"result": "TRUE"` e não termina. O driver libFuzzer do AFL++ não lê o caso de teste da stdin como o `NonDetGeneratorAFL.c`. Com B1, o replay do frontend precisa passar o arquivo como argumento. + +## Rota B sob KLEE (biblioteca nativa) + +O comando do brief não roda no KLEE 3.1: + +``` +klee: Unknown command line argument '--load=/src/spikes/llvm16/target-docker/release/libmap2check.so'. Try: 'klee --help' +klee: Did you mean '--libc=...'? +``` + +O KLEE 3.1 não tem mais `--load`. Correção de ambiente: pré-carregar a `.so` com `LD_PRELOAD`, já que o KLEE resolve chamadas externas por `dlsym` no próprio processo. Duas falhas no caminho: + +1. A `.so` com `--features libfuzzer` exige `__map2check_main__` na carga (`symbol lookup error: ... undefined symbol: __map2check_main__`), porque a ligação padrão do rustc (RELRO completo) resolve tudo na carga. Resolvido com `-C relro-level=off -C link-arg=-Wl,-z,lazy`. +2. A `.so` sem features não exporta `__VERIFIER_nondet_int` (0 símbolos): `KLEE: ERROR: failed external call: __VERIFIER_nondet_int`, 0 caminhos completos. + +Comando que roda: + +```bash +cargo rustc --release --features libfuzzer --lib --crate-type cdylib -- \ + -C panic=abort -C relro-level=off -C link-arg=-Wl,-z,lazy +LD_PRELOAD=$S klee --output-dir=klee-out-B --external-calls=all prog.bc +``` + +``` +KLEE: WARNING: undefined reference to function: __VERIFIER_nondet_int +KLEE: WARNING ONCE: calling external: __VERIFIER_nondet_int() at [no debug info] +KLEE: done: total instructions = 14 +KLEE: done: completed paths = 1 +KLEE: done: partially completed paths = 0 +KLEE: done: generated tests = 1 +``` + +`klee-stats --to-csv`: `Instructions=14, ... TerminationExit=1`. Para gerar a tabela, o `klee-stats` exige o pacote Python `tabulate`, ausente na imagem; o CSV não exige. + +Resultado: **1 caminho; `x == 42` não é explorado.** O nondet nativo devolve um valor concreto (0, sem buffer de fuzzer). Como o brief previa, a rota B não serve para `symex`. + +## Rota A′: rustc 1.72.1 (LLVM 16) + bitcode + +### A′ conforme o brief: crate `no_std` mínima (`spikes/llvm16/klee-rt`) + +`#![no_std]`, `panic = "abort"`, `#[global_allocator]` sobre `malloc`/`free`, `#[panic_handler]` com `abort`, um `Vec` de log (para exercitar `alloc`) e `__VERIFIER_nondet_int` chamando `klee_make_symbolic`. Perfil release com `lto = "fat"`. + +```bash +cd spikes/llvm16/klee-rt && RUSTUP_TOOLCHAIN=1.72.1 cargo rustc --release -- --emit=llvm-bc +llvm-dis-16 target/release/deps/klee_rt-*.bc # dis16=0 +llvm-link-16 ../prog.bc target/release/deps/klee_rt-*.bc -o linked.bc && klee linked.bc +``` + +``` +rustc 1.72.1 (d5c2e9c34 2023-09-13) LLVM version: 16.0.5 +warning: Linking two modules of different target triples: '...klee_rt-...bc' is 'x86_64-unknown-linux-gnu' whereas 'llvm-link' is 'x86_64-pc-linux-gnu' +KLEE: WARNING: undefined reference to function: rust_eh_personality +KLEE: WARNING ONCE: Alignment of memory from call "malloc" is not modelled. Using alignment of 8. +KLEE: done: total instructions = 350 +KLEE: done: completed paths = 2 +KLEE: done: partially completed paths = 0 +KLEE: done: generated tests = 2 +ktest: __VERIFIER_nondet_int int: 42 ktest: __VERIFIER_nondet_int int: 0 +``` + +Resultado: **passa, com 2 caminhos, como esperado.** O LTO é necessário. Sem ele (`CARGO_PROFILE_RELEASE_LTO=false`), o `--emit=llvm-bc` não inclui o código de `alloc`/`core`, e o KLEE para: + +``` +KLEE: WARNING: undefined reference to function: _ZN5alloc5alloc18handle_alloc_error... +KLEE: WARNING: undefined reference to variable: __rust_no_alloc_shim_is_unstable +KLEE: ERROR: Unable to load symbol(__rust_no_alloc_shim_is_unstable) while initializing globals +``` + +### A′ com `std` (`spikes/llvm16/klee-rt-std`, fora do brief) + +`Mutex>`, `println!`, LTO fat. O bitcode tem 391 definições e 73 declarações. Ele liga com `llvm-link-16`, mas o KLEE aborta: + +``` +inlinable function call in a function with debug info must have a !dbg location +LLVM ERROR: Broken module found, compilation aborted! +``` + +O debug info do `std` pré-compilado é incompatível com o verificador. Ao rodar `opt-16 --strip-debug` no bitcode, passa: + +``` +{"result":"TRUE","nondets":1} +KLEE: done: completed paths = 2 (sem libc; com --libc=uclibc --posix-runtime: idem, 15 371 instruções) +``` + +Restam 17 a 34 referências indefinidas, todas em caminhos que o programa não executa (`_Unwind_*`, `pthread_key_*`, `dl_iterate_phdr`, `getrandom`, `statx`). + +### A′ com a biblioteca real (`spikes/llvm16/full172`, fora do brief) + +Cópia de `Cargo.toml` + `rust/` com rustc 1.72.1. Obstáculos, em ordem: + +1. `error: failed to parse lock file at: /src/Cargo.lock`: o lockfile v4 não é lido pelo cargo 1.72. +2. Sem o lockfile, o resolvedor do 1.72 escolhe crates com `edition = "2024"` (`getrandom 0.4.3`, `wit-bindgen 0.57.1` via `mockall` e `cbindgen`). Correção: tirar `[dev-dependencies]` e `build.rs` (cbindgen) da cópia, declarar `rust-version = "1.72"` e gerar o lockfile com cargo 1.86 usando `CARGO_RESOLVER_INCOMPATIBLE_RUST_VERSIONS=fallback`, depois rebaixá-lo para `version = 3`. Resultado: `serde 1.0.229`, `serde_json 1.0.154`, `thiserror 2.0.20`. +3. Seis erros de fonte com o 1.72.1: + ``` + error: extern block cannot be declared unsafe --> rust/src/nondet/libfuzzer.rs:44 + error[E0658]: use of unstable library feature 'lazy_cell' --> rust/src/state.rs:4, :14, :15 (x2) + error[E0658]: use of unstable library feature 'mutex_unpoison' --> rust/src/state.rs:68 + ``` + Correção: `extern "C"` sem `unsafe`, `LazyLock` → `OnceLock::get_or_init` e remoção de `clear_poison()`. +4. Para o KLEE, um módulo `nondet/klee.rs` (feature `klee`) com `__VERIFIER_nondet_int` → `klee_make_symbolic`, que registra o valor no `AnalysisState` real. + +```bash +RUSTUP_TOOLCHAIN=1.72.1 cargo rustc --release --features klee --lib --crate-type staticlib -- --emit=llvm-bc -C panic=abort +opt-16 --strip-debug map2check-*.bc -o rt-nodbg.bc # só o runtime; o programa mantém -g +clang-16 -g -emit-llvm -c prog.c -o prog-g.bc +llvm-link-16 prog-g.bc rt-nodbg.bc -o linked5.bc +klee --external-calls=all --optimize --use-cex-cache --libc=uclibc linked5.bc # flags do caller.cpp:820 +``` + +``` +KLEE: done: total instructions = 68391 +KLEE: done: completed paths = 22 +KLEE: done: partially completed paths = 0 +KLEE: done: generated tests = 22 +``` + +Há 21 relatórios JSON `"result": "TRUE"` com `Container_NonDetLog` preenchido. Entre os ktests estão `42` (o caminho de `return 1`, sem relatório), `0`, `1`, `-8`, `-2147483648` etc. Sem o `--strip-debug` no runtime, o resultado é o mesmo `Broken module found` da variante `std`. + +**Explosão de caminhos:** a saída tem 22 caminhos, não 2. A serialização do valor simbólico pelo `serde_json` (formatação de inteiros) ramifica sobre o símbolo. O runtime C escreve o log por chamada externa, que concretiza o valor. O runtime Rust para KLEE precisa concretizar antes de formatar (`klee_get_value_i32`) ou não serializar valores simbólicos. + +Resultado: **a biblioteca real roda no KLEE 3.1/LLVM 16 pela rota A′**, com MSRV 1.72 (seis ajustes de fonte e `Cargo.lock` próprio), LTO fat, `--strip-debug` do runtime e concretização na saída. + +## Rota C: levar o Map2Check para LLVM 19 (pesquisa) + +```bash +cd Map2Check-cutover/modules/backend/pass +grep -ohE 'llvm::[A-Za-z:]+' *.cpp | sort -u | wc -l # 49 +grep -ohE 'llvm::[A-Za-z:]+' *.cpp *.hpp | sort -u | wc -l # 60 +``` + +São 9 passes, cerca de 4 250 linhas de C++, e 163 declarações `using llvm::…`. Usos que quebram ou mudam de semântica entre LLVM 16 e 19: + +| Uso | Contagem | Situação no LLVM 19 | +|---|---|---| +| `StringRef::startswith` / `endswith` | 2 + 4 | removidos (renomeados para `starts_with`/`ends_with`). Erro de compilação, troca mecânica | +| `DbgDeclareInst` / `dbg.declare` | 3 + 1 | o LLVM 19 representa debug info como *debug records* (RemoveDIs). Iterar instruções não acha mais `dbg.declare`. **Mudança de semântica**, exige revisão | +| `CreateGlobalStringPtr` | 25 | ainda existe no 19 (deprecado no 20) | +| `getOrInsertFunction` / `FunctionCallee` | 33 / 58 | estável | +| `PassPluginLibraryInfo` / `PassManager` (novo PM) | 10 / 10 | já no novo PM; estável | + +KLEE (releases no GitHub, consultadas em 2026-10-15): + +- **v3.1** (2024-02-29, a da imagem): LLVM 11–16. +- **v3.2** (2025-12-23): "Added partial support for LLVM 17-19"; recomendado LLVM 16. +- **v3.3** (2026-10-15, publicada hoje): "Added full support for LLVM 17-19", "Current recommended version is LLVM 18". + +Resposta: **sim, há KLEE com suporte a LLVM 19 (3.3)**, mas a release saiu hoje e não foi testada aqui. Não verifiquei se o AFL++ 4.40c compila com LLVM 19. Estimativa da Rota C: os passes são P–M (troca mecânica mais revisão do tratamento de debug info, cerca de 2 a 4 dias). Somam-se a reconstrução da imagem (clang/LLVM 19, KLEE 3.3 + STP/Z3 + klee-uclibc para LLVM 19, AFL++ recompilado) e a revalidação dos baselines de corpus, porque todo o pipeline muda de versão. No total, G (1 a 2 semanas), com o risco de ser a primeira versão de KLEE com suporte completo a LLVM 19. + +## Como reproduzir + +Na branch `spike/llvm16`: `spikes/llvm16/{b1build.sh,b2build.sh,fuzz.sh}`, `klee-rt/`, `klee-rt-std/`, `full172/` (com o `Cargo.lock` v3 usado) e `Dockerfile.rust`. From 482af559b240bdb28a444a7145f3034ea78fcf6c Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 20:18:48 -0400 Subject: [PATCH 07/63] docs: qualify D1 evidence after review (provisional afl criterion, CmpLog gap, real symex build, MSRV cost) --- docs/cutover/decisoes.md | 128 ++++++++++++++++++++++++++++------- docs/cutover/spike-llvm16.md | 100 ++++++++++++++++++++++----- 2 files changed, 186 insertions(+), 42 deletions(-) diff --git a/docs/cutover/decisoes.md b/docs/cutover/decisoes.md index e158b89..39f6a86 100644 --- a/docs/cutover/decisoes.md +++ b/docs/cutover/decisoes.md @@ -8,52 +8,130 @@ Registro das decisões D1–D5 do plano `docs/superpowers/plans/2026-10-15-cutov **Status: PROPOSTA — aguardando aprovação do mantenedor** -Evidência: `docs/cutover/spike-llvm16.md` (Task 1.1). Código na branch local `spike/llvm16`. +Evidência: `docs/cutover/spike-llvm16.md` (Task 1.1). Código na branch local `spike/llvm16` (commit `d13045b`). ### Proposta por gerador | Gerador | Rota proposta | Evidência principal | |---|---|---| -| `none` | **B**: `libmap2check.a` nativo (rustc do repo, 1.82) ligado no passo final, sem `llvm-link` | A rota B liga e roda com clang-16 (`rc=0`). O frontend de `develop` não executa o modo `none` (`Caller::executeAnalysis`: "This mode is not supported"; `map2check.cpp:977` troca `None` por AFL++). Basta que a ligação funcione, e ela usa o mesmo artefato do `afl`. | -| `afl` | **B + B1**: `libmap2check.a` nativo + driver libFuzzer do AFL++ (`afl-clang-fast -fsanitize=fuzzer`), chamando o `LLVMFuzzerTestOneInput` da biblioteca | Em 30 s, o B1 fez 97 474 exec/s contra 42 919 do `NonDetGeneratorAFL.c` (227%; 229% com CmpLog). Os dois acharam `x == 42`, com 1 `saved_crashes` cada. O critério da D1 (≥ 90% do C) é atendido sem portar o `__AFL_LOOP` para Rust (B2). | -| `symex` | **A′**: runtime compilado para bitcode por um rustc com LLVM 16 (1.72.1, LLVM 16.0.5), LTO fat, `opt-16 --strip-debug` só no runtime e `llvm-link-16` com o `-result.bc` | A rota A falha (`llvm-dis-16: error: Unknown attribute kind (91) (Producer: 'LLVM19.1.1-rust-1.82.0-stable' Reader: 'LLVM 16.0.6')`). A rota B sob KLEE dá 1 caminho, sem explorar `x == 42`. A A′ dá 2 caminhos (ktests 42 e 0) na crate `no_std` do brief. A **biblioteca real** (`std`, `serde_json`, `Mutex`) também roda no KLEE 3.1 com as flags do `caller.cpp`: 22 caminhos completos, `x == 42` encontrado. | +| `none` | **B**: `libmap2check.a` nativo (rustc do repo, 1.82), ligado no passo final, sem `llvm-link` | A rota B liga e roda com clang-16 (`rc=0`). O frontend de `develop` não executa `none` como gerador. `Caller::executeAnalysis` registra "This mode is not supported", e `map2check.cpp:977-1008` troca `None` por AFL++ → KLEE (→ AFL++ de novo com `--seed-exchange`). Na prática, `none` é a combinação de `afl` e `symex`, e herda as condições das duas, inclusive a 9. | +| `afl` | **B + B1**: `libmap2check.a` nativo + driver libFuzzer do AFL++ (`afl-clang-fast -fsanitize=fuzzer`), chamando o `LLVMFuzzerTestOneInput` da biblioteca | Os dois fuzzers acharam `x == 42` em 30 s, com 1 `saved_crashes` cada. Throughput: B1 97 474 exec/s, contra 42 919 do `NonDetGeneratorAFL.c` (B2), ou 227% (229% com CmpLog). **O critério da D1 (≥ 90% do C) está atendido provisoriamente.** A comparação não é igual por igual (condição 3). Com CmpLog, que é como o frontend sempre roda, o B1 levou **3 037 ms / 280 900 execs** até o crash, contra **252 ms / 2 748 execs** do B2 (condição 4). O B2, port do `__AFL_LOOP` para Rust, continua como alternativa nomeada se as condições 3 ou 4 falharem. | +| `symex` | **A′**: runtime compilado para bitcode por um rustc com LLVM 16 (1.72.1, LLVM 16.0.5), com manifesto e lockfile próprios, LTO fat e `opt-16 --strip-debug` só no runtime, depois `llvm-link-16` com o bitcode do programa | A rota A falha: `llvm-dis-16: error: Unknown attribute kind (91) (Producer: 'LLVM19.1.1-rust-1.82.0-stable' Reader: 'LLVM 16.0.6')`. A rota B sob KLEE dá 1 caminho e não explora `x == 42`. A A′ dá 2 caminhos (ktests 42 e 0) com a crate `no_std` do brief. A **biblioteca real** (`std`, `serde_json`, `Mutex`) roda no KLEE 3.1 com um subconjunto das flags do `caller.cpp`: 22 caminhos completos, `x == 42` encontrado. | -A rota C (Map2Check em LLVM 19) fica como **alternativa futura para `symex`**, não como caminho do cutover. O KLEE 3.3, com "full support for LLVM 17-19", saiu em 2026-10-15 e não foi testado. Migrar o pipeline inteiro custa G (1 a 2 semanas) mais a revalidação dos baselines. +A rota C (Map2Check em LLVM 19 + KLEE 3.3) fica como **saída futura do pin do rustc 1.72.1**, não como caminho do cutover (ver "Custo do pin e saída"). -O `symex` **não está bloqueado**: existe uma rota medida que funciona. A consequência prevista no brief para o caso de nenhuma rota servir não se aplica. O texto era: "o cutover completo fica bloqueado: pela restrição 'não remover sem equivalente validado', o runtime C do KLEE não pode sair". +O `symex` **não está bloqueado**, porque existe uma rota medida que funciona. A consequência que o brief previa para o caso de nenhuma rota servir não se aplica. O texto era: "o cutover completo fica bloqueado: pela restrição 'não remover sem equivalente validado', o runtime C do KLEE não pode sair". -### Comandos de build e ligação que a Task 3.1 implementa +### Comandos de build e ligação -`none` e `afl` (o passe já renomeia `main` para `__map2check_main__`): +#### `none` e `afl` + +O passe já renomeia `main` para `__map2check_main__`. ```bash -# biblioteca (rustc do repo) +# biblioteca (rustc do repo, 1.82) cargo build --release --features libfuzzer # -> target/release/libmap2check.a # programa: o -result.bc sai do llvm-link SEM os .bc do runtime C afl-clang-fast -g -O2 -fsanitize=fuzzer -result.bc libmap2check.a -ldl -lpthread -lm -o -fuzzed.out AFL_LLVM_CMPLOG=1 afl-clang-fast -g -O2 -fsanitize=fuzzer -result.bc libmap2check.a -ldl -lpthread -lm -o -cmplog.out afl-fuzz -i afl-in -o afl-out -c ./-cmplog.out -- ./-fuzzed.out -# replay: o crash vai como ARGUMENTO, não pela stdin (ver condição 2) +# replay: o crash vai como ARGUMENTO, não pela stdin (condição 2) ./-witness-fuzzed.out ``` -`symex`: +#### `symex` + +Estes são os comandos que produziram os 22 caminhos. Rodaram em `spikes/llvm16/full172/` na branch `spike/llvm16`, na imagem `map2check-dev:cutover-rust`. **Não funcionam no manifesto principal do repo.** Lá o cargo 1.72 falha com `error: failed to parse lock file at: /src/Cargo.lock` (lockfile v4). Sem esse lockfile, ele falha com `failed to parse the 'edition' key`, porque `mockall` e `cbindgen` puxam crates com `edition = "2024"`. ```bash -# runtime em bitcode LLVM 16 (toolchain separado, LTO fat no perfil) -RUSTUP_TOOLCHAIN=1.72.1 cargo rustc --release --features klee --lib --crate-type staticlib -- --emit=llvm-bc -C panic=abort -opt-16 --strip-debug target/release/deps/map2check-*.bc -o map2check-klee.bc -llvm-link-16 -output.bc map2check-klee.bc -o -witness-result.bc -klee --external-calls=all --optimize --use-cex-cache --libc=uclibc ./-witness-result.bc +cd spikes/llvm16/full172 +# Manifesto do build KLEE: cópia do Cargo.toml com `build = false` (sem cbindgen), +# sem [dev-dependencies] (sem mockall), `rust-version = "1.72"`, `[features] klee = []` +# e `[workspace]` próprio. Fontes: cópia de rust/ + nondet/klee.rs. + +# 1. lockfile próprio, resolvido por MSRV (cargo >= 1.84 para o resolvedor com fallback) +rustup toolchain install 1.86.0 --profile minimal +CARGO_RESOLVER_INCOMPATIBLE_RUST_VERSIONS=fallback RUSTUP_TOOLCHAIN=1.86.0 cargo generate-lockfile +sed -i 's/^version = 4/version = 3/' Cargo.lock # cargo 1.72 só lê v3 +# resultado: serde 1.0.229, serde_json 1.0.154, thiserror 2.0.20 + +# 2. runtime em bitcode LLVM 16; release com panic = "abort", lto = "fat", codegen-units = 1 +RUSTUP_TOOLCHAIN=1.72.1 CARGO_TARGET_DIR=/src/spikes/llvm16/target-172c \ + cargo rustc --release --features klee --lib --crate-type staticlib -- --emit=llvm-bc -C panic=abort +opt-16 --strip-debug /src/spikes/llvm16/target-172c/release/deps/map2check-*.bc -o rt-nodbg.bc + +# 3. ligação e execução (o programa mantém -g) +clang-16 -g -emit-llvm -c ../prog.c -o prog-g.bc +llvm-link-16 prog-g.bc rt-nodbg.bc -o linked5.bc +klee --external-calls=all --optimize --use-cex-cache --libc=uclibc linked5.bc +``` + +O frontend passa ainda `--solver-backend`, uma política de parada e de busca e a flag de sementes (`caller.cpp:815-822`). Essas flags não foram usadas no spike. + +**A Task 2.7 precisa transformar isso em produto.** Peças concretas: + +1. **Segundo manifesto** para o build KLEE, como membro de workspace ou diretório próprio (por exemplo `klee/Cargo.toml`). Ele aponta para as mesmas fontes de `rust/src`, com `build = false`, sem dev-dependencies e com `rust-version = "1.72"`. +2. **`Cargo.lock` v3 próprio** para esse manifesto, gerado com o comando acima e versionado. Ele é separado do lockfile v4 do build principal. +3. **Teto de dependências.** Só servem versões que resolvem e compilam em 1.72: hoje `serde_json ≤ 1.0.154` e `thiserror ≤ 2.0.20`. O `thiserror 2.0.21` já exige Rust 1.77. +4. **Feature `klee`**: código novo, inexistente no repo. A referência é `full172/rust/src/nondet/klee.rs`, com `__VERIFIER_nondet_int` → `klee_make_symbolic`. A Task 2.7 precisa escrever os demais tipos, `klee_assume` e a concretização da condição 7. +5. **LTO fat e `--strip-debug`** do bitcode do runtime. Sem LTO: `KLEE: ERROR: Unable to load symbol(__rust_no_alloc_shim_is_unstable)`. Com o debug info do `std`: `LLVM ERROR: Broken module found`. + +### Custo do pin do rustc 1.72.1 e saída + +Os 6 ajustes de fonte que o 1.72.1 exigiu, e o que cada um faz no build normal (rustc 1.82, `panic = unwind`, `ffi_guard` → `catch_unwind` em `rust/src/ffi.rs:100-107`): + +| Ajuste | Efeito no build normal | +|---|---| +| `unsafe extern "C"` → `extern "C"` (1 erro) | Nenhum: compila igual em 1.82 / edition 2021. | +| `LazyLock` → `OnceLock::get_or_init` (4 erros) | Nenhum: `OnceLock` existe desde 1.70, e a semântica de inicialização preguiçosa é a mesma. | +| Remover `state.clear_poison()` (1 erro) | **Muda comportamento.** Depois de um pânico capturado por `ffi_guard`, o mutex fica envenenado. `reset()` ainda recupera o guard com `into_inner`, mas não limpa o veneno. Todo `with_state` seguinte (`state.lock()?`) devolve `LockPoisoned`, até o fim de um processo persistente do AFL++. | + +**Para evitar a regressão,** o `clear_poison` continua no build normal e só sai do build `symex`: + +```rust +#[cfg(not(feature = "klee"))] +state.clear_poison(); ``` -### Condições da proposta (o que a aprovação aceita junto) +Sob `symex` isso é seguro, porque o perfil é `panic = "abort"` e não há pânico capturado que envenene o mutex. Há uma alternativa compatível com 1.72 sem `cfg`. É tornar `with_state` tolerante a veneno (`lock().unwrap_or_else(PoisonError::into_inner)`), como `state.rs:101` já faz nos testes. Ela muda o contrato de erro de `with_state`, então precisa de decisão própria. + +**Custos contínuos, aceitos junto com a A′ sobre a biblioteca real:** + +- (a) **O código que o build `symex` compila fica preso ao rustc 1.72 para sempre.** Ficam proibidos `LazyLock`, `unsafe extern`, edition 2024 e qualquer API de `std` posterior a 1.72. Isso precisa de um job de CI em 1.72.1, porque o `clippy` em 1.82 não detecta essas violações. +- (b) **Teto de dependências e dois lockfiles.** O v3 do `symex` e o v4 principal já divergem: `serde_json` 1.0.154 contra 1.0.150, `thiserror` 2.0.20 contra 2.0.18. O `cargo deny check` só cobre o lockfile principal, então o segundo precisa de passo próprio. Atualizar uma dependência passa a ser testado nos dois toolchains. +- (c) **O grafo do 1.72 não pode incluir `build.rs`/cbindgen nem `mockall`.** O header continua gerado só pelo build principal. + +**Alternativa que reduz o custo:** um crate `no_std` separado só para o KLEE, a A′ do brief, comprovada com 2 caminhos. Com ela, o crate principal fica livre em 1.82, e só um crate pequeno fica preso a 1.72. O preço é duplicar a lógica de estado e de saída que o KLEE precisa (log de nondet, resultado e relatório). Também perde-se a garantia de que KLEE e fuzzer rodam o mesmo código. A escolha entre "biblioteca real com MSRV 1.72 no que o `symex` compila" e "crate KLEE separado" fica com o mantenedor, como parte da aprovação desta D1. Esta proposta recomenda a biblioteca real com o `cfg` acima, porque um só código é o que torna o cutover verificável. + +**A saída do pin é a rota C** (LLVM 19 + KLEE 3.3, ). Vale a pena quando ocorrer o primeiro destes eventos: + +- o build principal precisar de algo que o 1.72 não compila e que o `cfg` não isola; +- o Map2Check decidir mudar de LLVM por outro motivo; +- o KLEE 3.3 (ou posterior) passar pelo corpus de baseline com resultados iguais aos do 3.1. + +Estimativa: G (1 a 2 semanas), com detalhes em `spike-llvm16.md`. + +### Condições da proposta + +A aprovação aceita a proposta junto com estas condições. + +1. **O `main` renomeado é obrigatório.** Com o `prog.c` cru, a ligação falha: `multiple definition of 'main'` no LibFuzzer, e `undefined reference to '__map2check_main__'` no LibFuzzer e no AFL++. O passe atual já renomeia o `main`, e isso não pode regredir. +2. **O replay pela stdin não funciona com B1.** `./b1.afl < crash` não termina (`rc=124` após 5 s), enquanto `./b1.afl crash` aborta (`rc=134`). O frontend (`caller.cpp:957`) hoje redireciona o arquivo para a stdin e precisa passar a usar argumento. Responsável: Task 3.x (frontend). +3. **Remedir o throughput de B1 contra B2, em condições iguais.** Responsável: Task 2.5, depois da Task 2.4. A comparação do spike não é igual por igual, por quatro motivos: + - o B2 grava arquivos a cada execução (`write_property` e `generate_aux_files` em `map2check_success`), e o B1 só imprime JSON na stdout; + - a instrumentação difere: o runtime C entra no `afl-clang-fast` como bitcode e é instrumentado; o `libmap2check.a` é objeto nativo e não é instrumentado; + - foi uma única rodada (n = 1); + - B1 e B2 rodaram ao mesmo tempo, na mesma máquina (`--cpus 4`). + + A remedição exige a saída em arquivo do Rust (Task 2.4), um `__VERIFIER_assume` que não encerre o processo persistente e programas com muitos `assume`. O Rust faz `exit(0)` (`libfuzzer.rs:197-206`); o C faz `longjmp` e segue para a próxima entrada. As campanhas devem rodar em sequência, com as mesmas flags e pelo menos 5 repetições. Abaixo de 90% do C, o `afl` passa para B2 (port do `__AFL_LOOP`). +4. **O tempo até o primeiro crash com CmpLog é critério de aceitação da D1.** Ele será remedido no corpus no gate 4.2. O frontend sempre roda o afl-fuzz com `-c `, então para guardas de valor mágico o que conta é o tempo até o crash, não exec/s. No spike, com CmpLog, o B1 levou cerca de 100× mais execuções que o B2 (280 900 contra 2 748), e o B2 achou o crash pela etapa de input-to-state (`op:its`). + + **Hipótese principal,** tirada do código e não medida (o spike não foi refeito): a diferença está em entradas mais curtas que o valor lido. O `read_bytes` do Rust (`libfuzzer.rs:54-71`) devolve `None` se faltarem bytes, e `read_u32` vira **0**. O C (`get_next_input_from_afl`) **volta ao início do buffer**. Com a semente `A` (1 byte), o C lê `x = 0x41414141`, que aparece na entrada. O CmpLog vê `0x41414141 == 42`, acha os bytes e os troca por `2a 00 00 00`. No Rust, `x = 0` não vem de nenhum byte da entrada, e o input-to-state não tem o que substituir. Só a mutação havoc, depois de crescer a entrada para 4 bytes, chega a `x = 42`. -1. **`main` renomeado é obrigatório.** Com o `prog.c` cru, a ligação falha (`multiple definition of 'main'` no LibFuzzer; `undefined reference to '__map2check_main__'` no LibFuzzer e no AFL++). O passe atual já renomeia o `main`; isso não pode regredir. -2. **Replay pela stdin não funciona com B1.** `./b1.afl < crash` não termina (`rc=124` após 5 s), enquanto `./b1.afl crash` aborta (`rc=134`). O frontend (`caller.cpp:957`) hoje redireciona o arquivo para a stdin e precisa passar a usar argumento. -3. **`assume` em persistent mode.** O C faz `longjmp` e segue para a próxima entrada. O Rust faz `exit(0)` (`libfuzzer.rs`, `__VERIFIER_assume`), que derruba o processo persistente. O throughput medido não cobre programas com `assume`. Medir na Task 2.5 antes de fechar o B1. Se cair abaixo de 90% do C, a resposta é B2 (port do loop). -4. **MSRV 1.72 para o build `symex`.** Com rustc 1.72.1, a biblioteca real tem 6 erros de fonte: `unsafe extern` (1), `LazyLock` (4) e `Mutex::clear_poison` (1). Também precisa de um `Cargo.lock` v3 resolvido por MSRV (`serde_json 1.0.154`, `thiserror 2.0.20`), sem `mockall`/`cbindgen` no grafo desse build. Trocas usadas no spike: `extern "C"`, `OnceLock`, sem `clear_poison`. Alternativa: um crate `no_std` separado só para KLEE (rota A′ do brief, comprovada), ao custo de duplicar a lógica de estado. -5. **LTO fat e `--strip-debug` no runtime `symex` são obrigatórios.** Sem LTO: `KLEE: ERROR: Unable to load symbol(__rust_no_alloc_shim_is_unstable)`. Com o debug info do `std`: `LLVM ERROR: Broken module found` ("inlinable function call in a function with debug info must have a !dbg location"). O `-g` do programa analisado é preservado. -6. **Concretizar antes de serializar no `symex`.** O `serde_json` formatando valores simbólicos levou `prog.c` de 2 para 22 caminhos. O gerador KLEE do Rust precisa concretizar (`klee_get_value_*`) o que vai para o log, como o C faz ao escrever por chamada externa. -7. **A rota B sob KLEE não é usada.** Além de não explorar símbolos, o KLEE 3.1 não tem `--load` (`Unknown command line argument '--load=...'`), e a `.so` só carrega com `LD_PRELOAD` e `-Wl,-z,lazy`. -8. **O `rustc 1.72.1` (2023) fica preso ao LLVM 16.** Quando o Map2Check migrar de LLVM, o toolchain do `symex` precisa acompanhar. A rota C remove essa dependência. + Hipótese secundária, do revisor: a leitura passa por código Rust não instrumentado. A correção provável da principal é a Task 2.5 alinhar a semântica de entrada curta à do C. Isso também é exigido pela condição 9. +5. **MSRV 1.72 no que o build `symex` compila,** com os custos (a)–(c) acima e o `clear_poison` sob `#[cfg(not(feature = "klee"))]`. Responsável: Task 2.7, com o job de CI em 1.72.1. +6. **LTO fat e `--strip-debug` do runtime `symex` são obrigatórios.** O `-g` do programa analisado é preservado. Responsável: Task 2.7. +7. **Concretizar antes de serializar no `symex`.** O `serde_json` formatando valores simbólicos levou o `prog.c` de 2 para 22 caminhos. O gerador KLEE do Rust precisa concretizar (`klee_get_value_*`) o que vai para o log e para o relatório, como o C faz ao escrever por chamada externa. Responsável: Task 2.7. +8. **A rota B sob KLEE não é usada.** Além de não explorar símbolos, o KLEE 3.1 não tem `--load` (`Unknown command line argument '--load=...'`), e a `.so` só carrega com `LD_PRELOAD` e `-Wl,-z,lazy`. +9. **O layout do vetor tem de ser o mesmo entre KLEE e AFL++.** No modo padrão (`none` → AFL++ → KLEE → AFL++ com `--seed-exchange`), os vetores do KLEE viram sementes do AFL++ (`Caller::exportKleeVectorsAsSeeds`). O gerador AFL do Rust tem de consumir a semente exatamente como o KLEE a produz: `sizeof(tipo)` bytes por valor, em ordem, little-endian. É o que `read_u32` faz hoje para `int`, mas falta verificar os demais tipos. A semântica de entrada curta precisa estar alinhada com a do C (condição 4). Responsável: Tasks 2.5 e 2.7, com teste cruzado ktest → semente → replay. +10. **O rustc 1.72.1 (2023) fica preso ao LLVM 16.** Quando o Map2Check migrar de LLVM, o toolchain do `symex` precisa acompanhar ou ser removido pela rota C. diff --git a/docs/cutover/spike-llvm16.md b/docs/cutover/spike-llvm16.md index a4ec7e3..ec8d7d3 100644 --- a/docs/cutover/spike-llvm16.md +++ b/docs/cutover/spike-llvm16.md @@ -47,11 +47,11 @@ Programas de prova (`spikes/llvm16/`): |---|---|---|---| | A: bitcode do rustc 1.82 no `llvm-link-16` | `dis16≠0` | `dis16=1`, `Unknown attribute kind (91)` | **falha (como esperado)** | | B: `libmap2check.a` nativo + `clang-16 -fsanitize=fuzzer` | liga e roda | com o `prog.c` cru, **não liga** (`main` duplicado + `__map2check_main__` indefinido); com `main` renomeado, liga e roda (`rc=0`) | **passa, desde que o passe renomeie `main`** | -| B1: staticlib + driver libFuzzer do AFL++ | throughput ≥ 90% do C | 97 474 exec/s, 1 crash (`x == 42`) em 0,39 s | **passa** (227% do B2) | +| B1: staticlib + driver libFuzzer do AFL++ | throughput ≥ 90% do C | 97 474 exec/s, 1 crash (`x == 42`) em 0,39 s; com CmpLog, 1º crash em 3,0 s / 280 900 execs (B2: 0,25 s / 2 748) | **passa provisoriamente** (227% do B2, comparação não igual por igual; CmpLog pior) | | B2: `NonDetGeneratorAFL.c` (referência) | — | 42 919 exec/s, 1 crash (`x == 42`) em 0,33 s | referência | | B sob KLEE (`.so` nativo) | 1 caminho, não explora `x == 42` | 1 caminho completo; `x == 42` não explorado | **falha para `symex` (como esperado)** | | A′: `no_std` + rustc 1.72.1 (LLVM 16) | 2 caminhos | 2 caminhos, ktests `42` e `0` | **passa** | -| A′ estendida: a biblioteca inteira (`std`, `serde_json`) com rustc 1.72.1 | — (fora do brief) | após 6 ajustes de fonte e `Cargo.lock` com resolução por MSRV, roda no KLEE com as flags do Map2Check: 22 caminhos, `x == 42` encontrado | **passa, com ressalvas** | +| A′ estendida: a biblioteca inteira (`std`, `serde_json`) com rustc 1.72.1 | — (fora do brief) | após 6 ajustes de fonte e `Cargo.lock` com resolução por MSRV, roda no KLEE com um subconjunto das flags do Map2Check: 22 caminhos, `x == 42` encontrado | **passa, com ressalvas** | | C: Map2Check em LLVM 19 | sim/não + estimativa | KLEE 3.3 (2026-10-15) tem suporte completo a LLVM 17–19; passes usam 49 APIs `llvm::` | **viável, não medido** | ## Rota A: bitcode do rustc 1.82 no LLVM 16 @@ -108,7 +108,20 @@ INFO: Loaded 1 modules (3 inline 8-bit counters) Done 1000 runs in 0 second(s) ``` -Cada execução imprime o relatório JSON (`"result": "TRUE"`) na stdout. Uma rodada de 2 000 000 execuções com `-use_value_profile=1` levou 7 s (285 714 exec/s). A instrumentação SanitizerCoverage entra no bitcode do usuário no passo final (3 contadores), não na biblioteca. +Cada execução imprime o relatório JSON (`"result": "TRUE"`) na stdout. Uma rodada de 2 000 000 execuções com `-use_value_profile=1`: + +```bash +./prog_m.fuzz -runs=2000000 -use_value_profile=1 -artifact_prefix=crB/ > fuzzB2.log 2>&1 # rc2=0 +grep -E "^#|Done|SUMMARY|deadly|crash|exit" fuzzB2.log | tail -5 +``` + +``` +#1048576 pulse cov: 2 ft: 67 corp: 58/229b lim: 4096 exec/s: 349525 rss: 28Mb +#2000000 DONE cov: 2 ft: 67 corp: 58/229b lim: 4096 exec/s: 285714 rss: 28Mb +Done 2000000 runs in 7 second(s) +``` + + A instrumentação SanitizerCoverage entra no bitcode do usuário no passo final (3 contadores), não na biblioteca. Resultado: **passa**, com a condição de que o `main` do programa seja renomeado pelo passe antes da ligação. @@ -133,7 +146,21 @@ afl-clang-fast -g -O2 -fsanitize=fuzzer prog42_m.bc $L -ldl -lpthread -lm -o b1. AFL_LLVM_CMPLOG=1 afl-clang-fast -g -O2 -fsanitize=fuzzer prog42_m.bc $L -ldl -lpthread -lm -o b1.cmplog.afl ``` -Liga. O binário tem `main` do driver, `LLVMFuzzerTestOneInput` da biblioteca, `__map2check_main__` e `__afl_persistent_loop`. Ele também contém o marcador `##SIG_AFL_PERSISTENT##`, então roda em persistent mode. Tamanho: 5 796 288 bytes (B2: 146 512). +Liga. Os símbolos foram verificados num build anterior com os mesmos fontes, mas sem `-g -O2` (`prog42_m.afl`). Para o `b1.afl` final, essa verificação não foi registrada: + +```bash +nm prog42_m.afl | grep -E " T (main|LLVMFuzzerTestOneInput|__map2check_main__|__afl_persistent_loop)$" +strings prog42_m.afl | grep -c SIG_AFL_PERSISTENT +``` + +``` +0000000000023830 T LLVMFuzzerTestOneInput +00000000000e5c70 T __afl_persistent_loop +00000000000188b0 T __map2check_main__ +1 +``` + +O `main` não aparece com o tipo `T`. Talvez seja o símbolo fraco do driver, mas isso não foi registrado. O marcador `##SIG_AFL_PERSISTENT##` está presente, então o binário roda em persistent mode. Tamanhos (`ls -la`): `b1.afl` tem 5 796 288 bytes; `b2.afl`, 146 512. ### B2: referência C (`NonDetGeneratorAFL.c`) @@ -166,9 +193,25 @@ Valores de `out/default/fuzzer_stats`: | B1 com CmpLog (`-c`) | 97 270,49 | 2 918 212 | 1 | 100,00% | 3 037 ms, 280 900 | sim | | B2 com CmpLog (`-c`) | 42 562,56 | 1 276 962 | 1 | 78,38% | 252 ms, 2 748 (`op:its`) | sim, entrada `2a 00 00 00` | -Throughput B1/B2: **227%** sem CmpLog e **229%** com CmpLog, acima do critério de 90% da D1. Ressalva: o B2 faz E/S de arquivo por execução (`write_property` e `generate_aux_files` em `map2check_success`). O B1 só imprime JSON na stdout, que o afl-fuzz descarta. Parte da diferença é custo de E/S, não do gerador. A comparação justa de throughput fica para quando a saída em arquivo do Rust existir (Task 2.4). +Throughput B1/B2: **227%** sem CmpLog e **229%** com CmpLog. O critério de 90% da D1 está **atendido só provisoriamente**, porque a comparação não é igual por igual: + +1. O B2 grava arquivos a cada execução (`write_property` e `generate_aux_files` em `map2check_success`). O B1 só imprime JSON na stdout, que o afl-fuzz descarta. +2. A instrumentação difere. O runtime C entra no `afl-clang-fast` como bitcode e é instrumentado junto com o programa. O `libmap2check.a` é objeto nativo e não é instrumentado. +3. Foi uma única rodada por variante (n = 1). +4. B1 e B2 rodaram ao mesmo tempo, no mesmo contêiner (`--cpus 4`). -Em ambos os casos o valor é lido como 4 bytes little-endian do início da entrada, então a semântica do vetor coincide nos dois runtimes. +A remedição em condições iguais é a condição 3 da D1 (`decisoes.md`). Ela vem depois da Task 2.4 (saída em arquivo do Rust) e da Task 2.5 (`assume`), e usa programas com muitos `assume`. + +**CmpLog e tempo até o crash.** O frontend sempre roda com CmpLog (`-c -cmplog.out`). Nesse modo, o B1 precisou de **3 037 ms e 280 900 execs** até o primeiro crash, e o B2 de **252 ms e 2 748 execs**: cerca de 100× mais execuções no B1. O crash do B2 veio da etapa de input-to-state (`op:its`); o do B1, de havoc. Sem CmpLog a diferença é pequena (27 994 contra 10 543 execs). Para guardas de valor mágico, comuns no SV-COMP, o tempo até o bug pesa mais que exec/s. + +- **Hipótese principal** (lida do código, não medida): entradas mais curtas que o valor lido. O `read_bytes` do Rust (`rust/src/nondet/libfuzzer.rs:54-71`) devolve `None` se faltarem bytes, e `read_u32` vira 0. O `get_next_input_from_afl` do C volta ao início do buffer. Com a semente `A` (1 byte), o C lê `x = 0x41414141`. O CmpLog vê `0x41414141 == 42`, acha os bytes `41` na entrada e os troca por `2a 00 00 00`. No Rust, `x = 0` não vem de nenhum byte da entrada, então o input-to-state não tem o que substituir. +- **Hipótese secundária:** a leitura passa por código Rust não instrumentado. + +O tempo até o primeiro crash com CmpLog passou a ser critério de aceitação da D1 (condição 4), remedido no corpus no gate 4.2. + +**Estabilidade.** O B2 ficou em 75,68% / 78,38%, e o B1 em 100%. Não investiguei a causa. Hipótese: o runtime C instrumentado carrega estado entre as iterações persistentes (contêineres alocados em `map2check_init`, arquivos, `map2check_destroy` com `static alreadyReleased`), e isso faz arestas variarem entre execuções da mesma entrada. O runtime Rust não é instrumentado e zera o estado a cada `LLVMFuzzerTestOneInput`. Por isso, mesmo que vazasse estado, isso não apareceria como instabilidade. + +Com entrada de pelo menos 4 bytes, os dois runtimes leem o valor como 4 bytes little-endian do início da entrada. Com entrada mais curta, divergem: o Rust devolve 0, e o C volta ao início do buffer (ver acima). ### Replay standalone do crash (achado relevante para a Task 3.x) @@ -265,29 +308,38 @@ O debug info do `std` pré-compilado é incompatível com o verificador. Ao roda KLEE: done: completed paths = 2 (sem libc; com --libc=uclibc --posix-runtime: idem, 15 371 instruções) ``` -Restam 17 a 34 referências indefinidas, todas em caminhos que o programa não executa (`_Unwind_*`, `pthread_key_*`, `dl_iterate_phdr`, `getrandom`, `statx`). +Restam referências indefinidas, todas em funções que o programa não executa. `klee3.log` é a execução sem libc; `klee4.log` usa `--libc=uclibc --posix-runtime`: + +```bash +grep -c "undefined reference" klee3.log # 34 +grep -c "undefined reference" klee4.log # 17 +``` + +Exemplos, nos dois logs: `_Unwind_Backtrace`, `_Unwind_GetIP`, `_Unwind_SetIP`, `__cxa_thread_atexit_impl`, `bcmp`, `dl_iterate_phdr` e `pthread_key_create`. Com a biblioteca real (`klee-full.log`), aparecem também `getrandom` e `statx`. ### A′ com a biblioteca real (`spikes/llvm16/full172`, fora do brief) Cópia de `Cargo.toml` + `rust/` com rustc 1.72.1. Obstáculos, em ordem: 1. `error: failed to parse lock file at: /src/Cargo.lock`: o lockfile v4 não é lido pelo cargo 1.72. -2. Sem o lockfile, o resolvedor do 1.72 escolhe crates com `edition = "2024"` (`getrandom 0.4.3`, `wit-bindgen 0.57.1` via `mockall` e `cbindgen`). Correção: tirar `[dev-dependencies]` e `build.rs` (cbindgen) da cópia, declarar `rust-version = "1.72"` e gerar o lockfile com cargo 1.86 usando `CARGO_RESOLVER_INCOMPATIBLE_RUST_VERSIONS=fallback`, depois rebaixá-lo para `version = 3`. Resultado: `serde 1.0.229`, `serde_json 1.0.154`, `thiserror 2.0.20`. +2. Sem o lockfile, o resolvedor do 1.72 escolhe crates com `edition = "2024"` (`getrandom 0.4.3`, `wit-bindgen 0.57.1` via `mockall` e `cbindgen`). Correção: tirar `[dev-dependencies]` e `build.rs` (cbindgen) da cópia, declarar `rust-version = "1.72"` e gerar o lockfile assim: `rustup toolchain install 1.86.0 --profile minimal && CARGO_RESOLVER_INCOMPATIBLE_RUST_VERSIONS=fallback RUSTUP_TOOLCHAIN=1.86.0 cargo generate-lockfile && sed -i 's/^version = 4/version = 3/' Cargo.lock`. Resultado: `serde 1.0.229`, `serde_json 1.0.154`, `thiserror 2.0.20`. 3. Seis erros de fonte com o 1.72.1: ``` error: extern block cannot be declared unsafe --> rust/src/nondet/libfuzzer.rs:44 error[E0658]: use of unstable library feature 'lazy_cell' --> rust/src/state.rs:4, :14, :15 (x2) error[E0658]: use of unstable library feature 'mutex_unpoison' --> rust/src/state.rs:68 ``` - Correção: `extern "C"` sem `unsafe`, `LazyLock` → `OnceLock::get_or_init` e remoção de `clear_poison()`. -4. Para o KLEE, um módulo `nondet/klee.rs` (feature `klee`) com `__VERIFIER_nondet_int` → `klee_make_symbolic`, que registra o valor no `AnalysisState` real. + Correção: `extern "C"` sem `unsafe`, `LazyLock` → `OnceLock::get_or_init` e remoção de `clear_poison()`. As duas primeiras compilam igual em 1.82. A terceira muda o comportamento do build normal, porque o mutex envenenado deixa de ser limpo. O custo e a mitigação (`#[cfg(not(feature = "klee"))]`) estão na D1, em `decisoes.md`. +4. Para o KLEE, um módulo novo `nondet/klee.rs` (feature `klee`, que não existe no repo e é só do spike) com `__VERIFIER_nondet_int` → `klee_make_symbolic`, que registra o valor no `AnalysisState` real. ```bash -RUSTUP_TOOLCHAIN=1.72.1 cargo rustc --release --features klee --lib --crate-type staticlib -- --emit=llvm-bc -C panic=abort -opt-16 --strip-debug map2check-*.bc -o rt-nodbg.bc # só o runtime; o programa mantém -g +cd spikes/llvm16/full172 # manifesto próprio: build = false, sem dev-deps, [workspace], Cargo.lock v3 +RUSTUP_TOOLCHAIN=1.72.1 CARGO_TARGET_DIR=/src/spikes/llvm16/target-172c \ + cargo rustc --release --features klee --lib --crate-type staticlib -- --emit=llvm-bc -C panic=abort +opt-16 --strip-debug /src/spikes/llvm16/target-172c/release/deps/map2check-*.bc -o rt-nodbg.bc # só o runtime; o programa mantém -g clang-16 -g -emit-llvm -c prog.c -o prog-g.bc llvm-link-16 prog-g.bc rt-nodbg.bc -o linked5.bc -klee --external-calls=all --optimize --use-cex-cache --libc=uclibc linked5.bc # flags do caller.cpp:820 +klee --external-calls=all --optimize --use-cex-cache --libc=uclibc linked5.bc # subconjunto das flags de caller.cpp:815-822 ``` ``` @@ -297,6 +349,8 @@ KLEE: done: partially completed paths = 0 KLEE: done: generated tests = 22 ``` +Ficaram de fora estas flags do frontend: `--solver-backend=` (ou `--solver-backend=metasmt --metasmt-backend=`), as políticas de parada e de busca, e a flag de sementes. + Há 21 relatórios JSON `"result": "TRUE"` com `Container_NonDetLog` preenchido. Entre os ktests estão `42` (o caminho de `return 1`, sem relatório), `0`, `1`, `-8`, `-2147483648` etc. Sem o `--strip-debug` no runtime, o resultado é o mesmo `Broken module found` da variante `std`. **Explosão de caminhos:** a saída tem 22 caminhos, não 2. A serialização do valor simbólico pelo `serde_json` (formatação de inteiros) ramifica sobre o símbolo. O runtime C escreve o log por chamada externa, que concretiza o valor. O runtime Rust para KLEE precisa concretizar antes de formatar (`klee_get_value_i32`) ou não serializar valores simbólicos. @@ -311,7 +365,19 @@ grep -ohE 'llvm::[A-Za-z:]+' *.cpp | sort -u | wc -l # 49 grep -ohE 'llvm::[A-Za-z:]+' *.cpp *.hpp | sort -u | wc -l # 60 ``` -São 9 passes, cerca de 4 250 linhas de C++, e 163 declarações `using llvm::…`. Usos que quebram ou mudam de semântica entre LLVM 16 e 19: +Os números de tamanho e de uso saem destes comandos, rodados no mesmo diretório: + +```bash +cat *.cpp *.hpp | wc -l # 4253 +grep -c "using llvm::\|namespace llvm" *.cpp *.hpp | awk -F: '{s+=$2} END{print s}' # 163 +grep -ohE '\b(startswith|endswith|CreateGlobalStringPtr|insertBefore|getTerminator|getOrInsertFunction|DebugLoc|dbg\.declare|DbgDeclareInst)\b' *.cpp *.hpp | sort | uniq -c | sort -rn +# 33 getOrInsertFunction 25 CreateGlobalStringPtr 5 DebugLoc 4 endswith +# 3 DbgDeclareInst 2 startswith 1 insertBefore 1 getTerminator 1 dbg.declare +grep -ohE '\b(FunctionCallee|PassManager|PassPluginLibraryInfo|PointerType::get)\b' *.cpp *.hpp | sort | uniq -c +# 58 FunctionCallee 10 PassManager 10 PassPluginLibraryInfo 44 PointerType::get +``` + +A contagem de passes vem do `CMakeLists.txt`, que tem 9 entradas em `MAP2CHECK_PASS_LIB`. Usos que quebram ou mudam de semântica entre LLVM 16 e 19: | Uso | Contagem | Situação no LLVM 19 | |---|---|---| @@ -321,11 +387,11 @@ São 9 passes, cerca de 4 250 linhas de C++, e 163 declarações `using llvm:: | `getOrInsertFunction` / `FunctionCallee` | 33 / 58 | estável | | `PassPluginLibraryInfo` / `PassManager` (novo PM) | 10 / 10 | já no novo PM; estável | -KLEE (releases no GitHub, consultadas em 2026-10-15): +KLEE: releases no GitHub, consultadas em 2026-10-15 com `curl -s https://api.github.com/repos/klee/klee/releases`. As frases entre aspas são citações literais do corpo de cada release: - **v3.1** (2024-02-29, a da imagem): LLVM 11–16. -- **v3.2** (2025-12-23): "Added partial support for LLVM 17-19"; recomendado LLVM 16. -- **v3.3** (2026-10-15, publicada hoje): "Added full support for LLVM 17-19", "Current recommended version is LLVM 18". +- **v3.2** (2025-12-23): "Added partial support for LLVM 17-19 (@mrexodia)" e "Current recommended version is LLVM 16". +- **v3.3** (`published_at` 2026-10-15T15:14:13Z, mesmo dia do spike): "Added full support for LLVM 17-19 (@ccadar)" e "Current recommended version is LLVM 18". Resposta: **sim, há KLEE com suporte a LLVM 19 (3.3)**, mas a release saiu hoje e não foi testada aqui. Não verifiquei se o AFL++ 4.40c compila com LLVM 19. Estimativa da Rota C: os passes são P–M (troca mecânica mais revisão do tratamento de debug info, cerca de 2 a 4 dias). Somam-se a reconstrução da imagem (clang/LLVM 19, KLEE 3.3 + STP/Z3 + klee-uclibc para LLVM 19, AFL++ recompilado) e a revalidação dos baselines de corpus, porque todo o pipeline muda de versão. No total, G (1 a 2 semanas), com o risco de ser a primeira versão de KLEE com suporte completo a LLVM 19. From 3f7101f689873f1743509ed78b05aef0886dbfe9 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 20:37:12 -0400 Subject: [PATCH 08/63] docs: record maintainer approval of the runtime integration route (D1) --- docs/cutover/decisoes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cutover/decisoes.md b/docs/cutover/decisoes.md index 39f6a86..3135056 100644 --- a/docs/cutover/decisoes.md +++ b/docs/cutover/decisoes.md @@ -6,7 +6,7 @@ Registro das decisões D1–D5 do plano `docs/superpowers/plans/2026-10-15-cutov **Pergunta:** como o runtime Rust entra no pipeline LLVM 16 do Map2Check (`develop`: clang/LLVM 16.0.6, AFL++ 4.40c, KLEE 3.1), e como o AFL++ dirige as entradas. -**Status: PROPOSTA — aguardando aprovação do mantenedor** +**Status: APROVADA pelo mantenedor em 2026-10-15** (aprovação dada na sessão de execução do plano; as condições abaixo continuam obrigatórias para as tasks que as possuem) Evidência: `docs/cutover/spike-llvm16.md` (Task 1.1). Código na branch local `spike/llvm16` (commit `d13045b`). From bd1d91726e049f4a2e0a60a3e47183ec03025f49 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 20:39:08 -0400 Subject: [PATCH 09/63] build: add unwinding runtime profile and klee feature --- Cargo.toml | 8 ++++++++ tests/runtime_profile.rs | 22 ++++++++++++++++++++++ 2 files changed, 30 insertions(+) create mode 100644 tests/runtime_profile.rs diff --git a/Cargo.toml b/Cargo.toml index ba4ff38..86194ab 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -22,6 +22,7 @@ mockall = "0.13" [features] libfuzzer = [] +klee = [] [profile.release] opt-level = 3 @@ -32,5 +33,12 @@ codegen-units = 1 opt-level = 0 debug = true +# Unwind e não abort: ffi_guard (rust/src/ffi.rs) recupera panics com +# catch_unwind, e um abort sob AFL++ persistente pareceria crash do programa +# analisado. O build KLEE usa panic = "abort" separadamente. +[profile.runtime] +inherits = "release" +panic = "unwind" + [build-dependencies] cbindgen = "0.26.0" \ No newline at end of file diff --git a/tests/runtime_profile.rs b/tests/runtime_profile.rs new file mode 100644 index 0000000..91666d6 --- /dev/null +++ b/tests/runtime_profile.rs @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-3.0-only +// O runtime ligado ao programa analisado precisa desenrolar panics através de +// catch_unwind (ffi_guard), e não abortar o processo: um abort sob AFL++ +// persistente apareceria como crash do programa analisado (falso positivo). +// O build KLEE usa abort em separado, numa task posterior. +#[test] +fn runtime_profile_unwinds_on_panic() { + let manifest = + std::fs::read_to_string(concat!(env!("CARGO_MANIFEST_DIR"), "/Cargo.toml")).unwrap(); + let table = manifest.split("[profile.runtime]").nth(1).expect( + "perfil runtime ausente: adicione [profile.runtime] com inherits = \"release\" e panic = \"unwind\" ao Cargo.toml", + ); + let table = table.split("\n[").next().unwrap(); + assert!( + table.contains("inherits = \"release\""), + "[profile.runtime] deve herdar de release: {table}" + ); + assert!( + table.contains("panic = \"unwind\""), + "[profile.runtime] deve declarar panic = \"unwind\" (ffi_guard depende de catch_unwind): {table}" + ); +} From b0240160fdc3d2f38a7ad51cba79f3cd9817b3d0 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 20:42:14 -0400 Subject: [PATCH 10/63] fix: reset analysis state on every init and fuzz iteration (#9) --- rust/src/ffi_tests.rs | 32 +++++++++++++++ rust/src/nondet/libfuzzer.rs | 75 +++++++++++++++++++++++++++++++++++- rust/src/state.rs | 5 ++- 3 files changed, 110 insertions(+), 2 deletions(-) diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 16e2a1c..428340c 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -55,6 +55,38 @@ fn init_leaves_state_empty_and_ok() { && s.bbtrack.is_empty())); } +#[test] +fn second_init_starts_from_clean_state() { + let _guard = fresh(); + map2check_init(); + // SAFETY: fake addresses are only stored, never dereferenced. + unsafe { map2check_map_malloc(addr(0x1000), 8) }; + // SAFETY: C-string literal is valid and null-terminated. + unsafe { set_false_result(5, 7, c"main".as_ptr()) }; + map2check_set_memcleanup(); + map2check_set_null_is_valid(); + let mut value: i32 = 3; + // SAFETY: `value` outlives the call; the name is a valid C string. + unsafe { map2check_save_nondet_log_int(1, 0, 0, &mut value, c"main".as_ptr()) }; + let _ = get_next_step(); + assert!(snapshot(|s| s.current_step() > 0 && !s.nondets.is_empty())); + + map2check_init(); + + assert_ok(); + snapshot(|s| { + assert_eq!(s.current_step(), 0); + assert!(s.memtrack.is_empty()); + assert!(s.nondets.is_empty()); + assert!(s.bbtrack.is_empty()); + assert!(s.legacy_nondet_logs.is_empty()); + assert!(!s.memcleanup_enabled); + assert!(!s.null_is_valid); + assert_eq!(s.result.line, 0); + assert!(s.result.function_name.is_empty()); + }); +} + #[test] fn reset_clears_records_result_step_and_flags() { let _guard = fresh(); diff --git a/rust/src/nondet/libfuzzer.rs b/rust/src/nondet/libfuzzer.rs index 193aa68..bf4c69e 100644 --- a/rust/src/nondet/libfuzzer.rs +++ b/rust/src/nondet/libfuzzer.rs @@ -255,9 +255,21 @@ pub extern "C" fn __VERIFIER_assert(condition: i32) { #[cfg(test)] #[no_mangle] pub extern "C" fn __map2check_main__() -> i32 { - 0 // Retorna 0 (sucesso) e não faz nada + // Runs the per-test hook (if any); otherwise succeeds and does nothing. + let hook = *TEST_MAIN_HOOK + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + if let Some(hook) = hook { + hook(); + } + 0 } +/// Body of the fake `__map2check_main__`, installed by a test while it holds +/// the global test lock. +#[cfg(test)] +static TEST_MAIN_HOOK: std::sync::Mutex> = std::sync::Mutex::new(None); + #[cfg(test)] mod tests { use super::*; @@ -376,6 +388,67 @@ mod tests { assert_eq!(FUZZER_CURSOR.load(Ordering::SeqCst), 0); } + /// Fake program body: tracks one allocation and one nondet value, and + /// records the first byte of its input. + fn iteration_body() { + // SAFETY: fake addresses are only stored, never dereferenced. + unsafe { crate::ffi::map2check_map_malloc(0x1000 as *const _, 8) }; + let _ = __VERIFIER_nondet_uchar(); + let first = crate::state::with_state(|s| { + s.nondets.push(crate::nondet::NonDetEntry::new( + 1, + 1, + 0, + "main", + crate::nondet::NonDetValue::Int(7), + )); + Ok(s.current_step()) + }) + .unwrap(); + ITERATION_STEPS.fetch_add(first as usize, Ordering::SeqCst); + } + + static ITERATION_STEPS: AtomicUsize = AtomicUsize::new(0); + + fn set_hook(hook: Option) { + *TEST_MAIN_HOOK.lock().unwrap() = hook; + } + + #[test] + fn two_fuzzer_iterations_yield_independent_results() { + let _guard = crate::state::test_lock(); + set_hook(Some(iteration_body)); + ITERATION_STEPS.store(0, Ordering::SeqCst); + + let first = [0xAA_u8, 1, 2]; + // SAFETY: buffers are valid for their lengths for the whole call. + unsafe { LLVMFuzzerTestOneInput(first.as_ptr(), first.len()) }; + // Dirty the state between iterations as a crashed/violating run would. + crate::state::with_state(|s| { + s.set_false(crate::caller::ViolatedProperty::Overflow, 9, "first"); + s.next_step(); + Ok(()) + }) + .unwrap(); + let second = [0xBB_u8]; + // SAFETY: as above. + unsafe { LLVMFuzzerTestOneInput(second.as_ptr(), second.len()) }; + set_hook(None); + + crate::state::with_state(|s| { + assert!(s.result.ok, "first-run violation leaked into the second"); + assert_eq!(s.result.line, 0); + assert!(s.result.function_name.is_empty()); + // The second body ran one step-counting call (nondet_uchar) itself. + assert_eq!(s.current_step(), 1, "step counter leaked"); + assert_eq!(s.memtrack.len(), 1, "first-run allocation leaked"); + assert_eq!(s.nondets.len(), 1, "first-run nondet log leaked"); + Ok(()) + }) + .unwrap(); + assert_eq!(FUZZER_CURSOR.load(Ordering::SeqCst), 0); + } + #[test] fn fuzzer_iteration_rejects_null_data_with_nonzero_size() { let _guard = crate::state::test_lock(); diff --git a/rust/src/state.rs b/rust/src/state.rs index 9c5bf35..79a80d9 100644 --- a/rust/src/state.rs +++ b/rust/src/state.rs @@ -54,8 +54,11 @@ fn ensure_state() -> &'static Mutex { &ANALYSIS_STATE } +/// Starts a fresh analysis: forces the singleton into existence and clears +/// every counter, container, result and flag, so a second `map2check_init()` +/// in the same process never sees data from an earlier analysis. pub fn init() { - let _ = ensure_state(); + reset(); } pub fn reset() { From 4899d7300a7edf2692dd11a7f1ac624ed21f9a1f Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 20:43:19 -0400 Subject: [PATCH 11/63] fix: guard every FFI export, document panic policy and test pre-init calls (#8) --- docs/api-deltas.md | 16 ++++++ rust/src/ffi.rs | 14 +++++ rust/src/nondet/libfuzzer.rs | 2 +- tests/ffi_guard_coverage.rs | 107 +++++++++++++++++++++++++++++++++++ tests/ffi_harness.c | 95 +++++++++++++++++++++++++++++++ 5 files changed, 233 insertions(+), 1 deletion(-) create mode 100644 tests/ffi_guard_coverage.rs diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 44264a5..73d4651 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -176,6 +176,22 @@ exercises. `__VERIFIER_nondet_*`-style generator for the specific width needed instead of resurrecting the old kernel-typed names. +## Accepted: failure behaviour at the FFI boundary (panic strategy) + +Every `#[no_mangle]` export runs its body through `ffi_guard` +(`rust/src/ffi.rs`), enforced by the source-scanning test +`tests/ffi_guard_coverage.rs`. + +| Build | Panic strategy | Behaviour on a Rust panic inside an export | +|---|---|---| +| `dev`, `release`, `runtime` | `unwind` | Caught by `catch_unwind` in `ffi_guard`; a diagnostic goes to stderr and the export returns its neutral default. Nothing unwinds into C. | +| any `panic = "abort"` build (future KLEE build) | `abort` | No recovery: the process aborts at the panic. `ffi_guard` is then a no-op wrapper. | + +Deliberate termination (`__VERIFIER_error` aborts, `__VERIFIER_assume(0)` +exits with 0) is not a failure and is unaffected. `tests/ffi_harness.c` +calls every export before `map2check_init()` with valid boundary inputs +under ASan/UBSan. + ## Accepted: `__VERIFIER_assume` / `map2check_fuzzer_assume` end the process `__VERIFIER_assume(0)` and `map2check_fuzzer_assume(0)` call diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index 94dd6ea..72292bb 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -97,6 +97,20 @@ fn violated_property_from_raw(raw: i32) -> Option { } } +/// Failure policy at the `extern "C"` boundary (issue #8). +/// +/// Every `#[no_mangle]` export runs its body through this function +/// (enforced by `tests/ffi_guard_coverage.rs`). +/// +/// - Unwinding profiles (`dev`, `release`, `runtime`; `panic = "unwind"`): +/// a panic is caught here with `catch_unwind`, a diagnostic is written to +/// stderr and `default` is returned, so no unwind ever reaches C. +/// - `panic = "abort"` builds (the future KLEE build): `catch_unwind` cannot +/// intercept anything; a panic aborts the process immediately and there is +/// no recovery. Exports therefore must not rely on panics for control flow. +/// +/// Deliberate process termination (`__VERIFIER_error`, `__VERIFIER_assume(0)`) +/// is not a failure and bypasses recovery by design. pub(crate) fn ffi_guard(f: impl FnOnce() -> T, default: T) -> T { match std::panic::catch_unwind(std::panic::AssertUnwindSafe(f)) { Ok(value) => value, diff --git a/rust/src/nondet/libfuzzer.rs b/rust/src/nondet/libfuzzer.rs index bf4c69e..0816ef9 100644 --- a/rust/src/nondet/libfuzzer.rs +++ b/rust/src/nondet/libfuzzer.rs @@ -210,7 +210,7 @@ pub extern "C" fn __VERIFIER_assume(condition: i32) { /// generator. See `__VERIFIER_assume` for the exit(0) design note. #[no_mangle] pub extern "C" fn map2check_fuzzer_assume(condition: i32) { - __VERIFIER_assume(condition); + crate::ffi::ffi_guard(|| __VERIFIER_assume(condition), ()); } /// SV-COMP reachability primitive: records a `Reachability` violation and diff --git a/tests/ffi_guard_coverage.rs b/tests/ffi_guard_coverage.rs new file mode 100644 index 0000000..ab22411 --- /dev/null +++ b/tests/ffi_guard_coverage.rs @@ -0,0 +1,107 @@ +// SPDX-License-Identifier: GPL-3.0-only +//! Issue #8: every `#[no_mangle]` export must run its body inside +//! `ffi_guard`, so a panic is recovered at the FFI boundary (on unwinding +//! profiles) instead of unwinding into C. + +mod common; + +use common::read; + +const SOURCES: &[&str] = &["rust/src/ffi.rs", "rust/src/nondet/libfuzzer.rs"]; + +/// Drops comments and string/char-literal contents so braces inside them do +/// not confuse the body matcher. +fn strip(src: &str) -> String { + let c: Vec = src.chars().collect(); + let mut out = String::new(); + let mut i = 0; + while i < c.len() { + if c[i] == '/' && c.get(i + 1) == Some(&'/') { + while i < c.len() && c[i] != '\n' { + i += 1; + } + } else if c[i] == '"' { + i += 1; + while i < c.len() && c[i] != '"' { + i += if c[i] == '\\' { 2 } else { 1 }; + } + i += 1; + out.push_str("\"\""); + } else if c[i] == '\'' && c.get(i + 2) == Some(&'\'') { + i += 3; + } else { + out.push(c[i]); + i += 1; + } + } + out +} + +/// Returns `(name, body)` for each `#[no_mangle]` function that is not +/// test-only. +fn exports(src: &str) -> Vec<(String, String)> { + let mut found = Vec::new(); + let mut rest = src; + while let Some(pos) = rest.find("#[no_mangle]") { + let before = &rest[..pos]; + let after = &rest[pos + "#[no_mangle]".len()..]; + rest = after; + // `#[cfg(test)]` directly above the attribute marks a test double. + if before.trim_end().ends_with("#[cfg(test)]") { + continue; + } + let fn_at = after.find("fn ").expect("fn after #[no_mangle]") + 3; + let name: String = after[fn_at..] + .chars() + .take_while(|ch| ch.is_alphanumeric() || *ch == '_' || *ch == '$') + .collect(); + let open = after.find('{').expect("function body"); + let mut depth = 0; + let mut end = open; + for (off, ch) in after[open..].char_indices() { + match ch { + '{' => depth += 1, + '}' => { + depth -= 1; + if depth == 0 { + end = open + off; + break; + } + } + _ => {} + } + } + found.push((name, after[open + 1..end].to_owned())); + } + found +} + +#[test] +fn every_no_mangle_export_body_is_wrapped_in_ffi_guard() { + let mut total = 0; + for file in SOURCES { + let src = strip(&read(file)); + // Only production code: drop the in-file test module. + let prod = src.split("#[cfg(test)]\nmod tests").next().unwrap(); + for (name, body) in exports(prod) { + total += 1; + let trimmed = body.trim(); + assert!( + trimmed.starts_with("ffi_guard(") || trimmed.contains("ffi_guard("), + "{file}: export `{name}` is not wrapped in ffi_guard" + ); + } + } + assert!( + total > 40, + "scanner found only {total} exports; is it broken?" + ); +} + +#[test] +fn scanner_detects_an_unguarded_export() { + let src = "#[no_mangle]\npub extern \"C\" fn bad() { panic!(); }\n"; + let found = exports(&strip(src)); + assert_eq!(found.len(), 1); + assert!(!found[0].1.contains("ffi_guard(")); +} diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index 01045b4..71c601f 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -13,6 +13,7 @@ */ #include #include +#include #include "../include/map2check.h" @@ -29,6 +30,97 @@ int __map2check_main__(void) { return 0; } +/* ── Round 0 (issue #8): every export called BEFORE map2check_init(), as the + * very first thing the process does, with valid boundary inputs only: NULL + * where a pointer is allowed, real stack/heap addresses, valid C strings and + * zero sizes. No arbitrary invalid pointers are ever passed (they could fault + * before Rust gets a chance to validate them). ─────────────────────────── */ +static void run_pre_init_boundary(void) { + int32_t int_value = 1; + uint32_t uint_value = 1; + long long_value = 1; + char char_value = 'x'; + float float_value = 0.5f; + double double_value = 0.5; + int stack_obj = 0; + void *heap_obj = malloc(8); + if (heap_obj == NULL) { + return; + } + + set_false_result(None, 0, ""); + (void)get_current_property(); + (void)get_current_step(); + (void)get_next_step(); + vcc_reset_meta_data(); + map2check_set_memcleanup(); + map2check_set_null_is_valid(); + + map2check_map_alloca("", &stack_obj, 0, 0, 1, 0); + map2check_map_non_static_alloca("", &stack_obj, 0, 0, 1, 0); + map2check_map_funct_address("main", &stack_obj); + map2check_map_malloc(heap_obj, 0); + map2check_map_calloc(heap_obj, 0, 0); + map2check_map_store_pointer(&stack_obj, heap_obj, 0, "", 1, "main"); + map2check_check_load(heap_obj, 1, 0, 0, "main"); + map2check_check_deref(heap_obj, 0, 1, "main"); + map2check_check_free("", heap_obj, 0, 1, "main"); + map2check_check_free_resolved_address(heap_obj, 1, "main", 0); + map2check_map_free("", heap_obj, 0, 1, "main"); + map2check_check_mem_endprog(); + + (void)map2check_save_basic_block_log(1, "main"); + (void)map2check_is_in_trackbb_container(0); + + (void)map2check_save_nondet_log_int(1, 0, INT_ID, &int_value, "main"); + (void)map2check_save_nondet_log_uint(1, 0, UNIT_ID, &uint_value, "main"); + (void)map2check_save_nondet_log_long(1, 0, LONG_ID, &long_value, "main"); + (void)map2check_save_nondet_log_char(1, 0, CHAR_ID, &char_value, "main"); + (void)map2check_save_nondet_log_float(1, 0, FLOAT_ID, &float_value, "main"); + (void)map2check_save_nondet_log_double(1, 0, DOUBLE_ID, &double_value, "main"); + + map2check_is_valid_assert(1, "main", 1); + map2check_binop_add_int(1, 1, 1, 0, "main"); + map2check_binop_sub_int(1, 1, 1, 0, "main"); + map2check_binop_mul_int(1, 1, 1, 0, "main"); + map2check_binop_div_int(1, 1, 1, 0, "main"); + map2check_binop_add_unsigned(1u, 1u, 1, 0, "main"); + map2check_binop_sub_unsigned(1u, 1u, 1, 0, "main"); + map2check_binop_mul_unsigned(1u, 1u, 1, 0, "main"); + map2check_binop_div_unsigned(1u, 1u, 1, 0, "main"); + map2check_binop_neg_int(1, 0, 1, 0, "main"); + map2check_binop_shl_int(1, 1, 1, 0, "main"); + map2check_binop_shr_int(1, 1, 1, 0, "main"); + map2check_binop_shr_unsigned(1u, 1u, 1, 0, "main"); + + (void)__VERIFIER_nondet_bool(); + (void)__VERIFIER_nondet_char(); + (void)__VERIFIER_nondet_uchar(); + (void)__VERIFIER_nondet_short(); + (void)__VERIFIER_nondet_ushort(); + (void)__VERIFIER_nondet_int(); + (void)__VERIFIER_nondet_uint(); + (void)__VERIFIER_nondet_long(); + (void)__VERIFIER_nondet_ulong(); + (void)__VERIFIER_nondet_float(); + (void)__VERIFIER_nondet_double(); + (void)__VERIFIER_nondet_size_t(); + (void)__VERIFIER_nondet_pointer(); + + /* truthy conditions only: falsy ones terminate the process by design */ + __VERIFIER_assume(1); + __VERIFIER_assert(1); + map2check_fuzzer_assume(1); + + /* LLVMFuzzerTestOneInput with a zero-size, NULL buffer is valid. */ + (void)LLVMFuzzerTestOneInput(NULL, 0); + + print_all_containers_as_json(); + map2check_success(); + map2check_reset(); + free(heap_obj); +} + /* ── Round 1: hostile calls made *before* map2check_init() ────────────── * Exercises null pointers, empty/garbage strings and pre-init access for * every export that can plausibly be called out of order. @@ -254,6 +346,9 @@ int main(void) { return 1; } + /* round 0: all exports before map2check_init() (issue #8) */ + run_pre_init_boundary(); + if (test_result_and_control_flags() != 0) { return 1; } From daa3f616abac64ec0d92639a4e6c8d7352bf0ecd Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 20:51:20 -0400 Subject: [PATCH 12/63] fix: keep the first violation when later checks pass (#36) A memory check that found the access safe reset the result to OK, which erased any earlier violation, including one recorded by set_false_result. The C runtime of Map2Check develop ends the process on the first violation (map2check_error -> map2check_exit -> abort), so the first violation is final there. AnalysisState::set_false now ignores every violation after the first, and the memory checks no longer reset the result before checking. The analysis keeps running; ending the process belongs to a later task. Invert the pinning test into sd03_safe_check_keeps_earlier_violation, add sd03_later_violation_does_not_overwrite_first, split the tests that chained independent violations, flip the harness case, mark SD-03 fixed (new 'fixed' classification) and update the integration guide. --- docs/api-deltas.md | 4 ++-- docs/integration-guide.md | 7 ++++--- docs/semantic-deltas.md | 43 +++++++++++++++++++++++++------------- rust/src/ffi.rs | 17 --------------- rust/src/ffi_tests.rs | 44 +++++++++++++++++++++++++++++++++++---- rust/src/state.rs | 9 ++++++++ tests/compat_docs.rs | 3 ++- tests/ffi_harness.c | 5 ++++- 8 files changed, 90 insertions(+), 42 deletions(-) diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 73d4651..9c409b0 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -46,7 +46,7 @@ build with `--features libfuzzer`. | `map2check_init` | `void map2check_init(void)` | `map2check_init` | same | Map2Check passes call `map2check_init(int)`; see the pass matrix. | | `map2check_reset` | `void map2check_reset(void)` | — | new | Clears containers, result, step counter and flags (#9). | | `map2check_success` | `void map2check_success(void)` | `map2check_success` | same | Prints the JSON report (SD-01). | -| `set_false_result` | `void set_false_result(int32_t prp, int32_t line_number, const char *function_name)` | `set_false_result` | same | `ViolatedProperty` keeps the C variant order (`caller_lib_result.h:22-31`); unknown values are ignored (SD-09). | +| `set_false_result` | `void set_false_result(int32_t prp, int32_t line_number, const char *function_name)` | `set_false_result` | same | `ViolatedProperty` keeps the C variant order (`caller_lib_result.h:22-31`); unknown values are ignored (SD-09). Ignored when a violation is already recorded (SD-03). | | `get_current_property` | `enum ViolatedProperty get_current_property(void)` | `get_current_property` | same | | | `get_current_step` | `uint64_t get_current_step(void)` | `long get_current_step(void)` | changed | Returns `uint64_t` instead of `long`. | | `get_next_step` | `uint64_t get_next_step(void)` | `get_next_step()` / `next_current_step()` | same | Advances and returns the step counter. `next_current_step` is not exported; see its row. | @@ -82,7 +82,7 @@ build with `--features libfuzzer`. | `map2check_binop_mul_unsigned` | `void map2check_binop_mul_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_mul_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | | `map2check_binop_div_unsigned` | `void map2check_binop_div_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_div_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | | `map2check_binop_shr_unsigned` | `void map2check_binop_shr_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shr_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_check_load` | `void map2check_check_load(const void *ptr, int32_t line, uint32_t scope, int32_t size, const char *function_name)` | `map2check_check_load` | changed | Address-validity rule rewritten (plan R1; differential check in #15). A safe load clears an earlier violation (SD-03); NULL handling in SD-05. | +| `map2check_check_load` | `void map2check_check_load(const void *ptr, int32_t line, uint32_t scope, int32_t size, const char *function_name)` | `map2check_check_load` | changed | Address-validity rule rewritten (plan R1; differential check in #15). The first violation is final; a later check never clears or replaces it (SD-03); NULL handling in SD-05. | | `map2check_check_free` | `void map2check_check_free(const char *_name, const void *ptr, uint32_t _scope, uint32_t line, const char *function_name)` | `map2check_check_free` | same | `_name` and `_scope` are ignored. Untracked address is a violation (SD-06). | | `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; untracked address is safe (SD-06). | | `map2check_check_deref` | `void map2check_check_deref(const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_deref` | same | Name collides with the pass ABI `map2check_check_deref(i64, ptr)`; see the pass matrix (#21). | diff --git a/docs/integration-guide.md b/docs/integration-guide.md index 624f362..2af7513 100644 --- a/docs/integration-guide.md +++ b/docs/integration-guide.md @@ -80,9 +80,10 @@ cc -I include my_program.c lib/libmap2check.a -ldl -lpthread -lm -o my_program 6. `map2check_reset()` to analyse another input in the same process. LibFuzzer builds do this on every iteration. -A violation does not stop the program, and a later safe memory check -resets the verdict (SD-03). Read the verdict right after the check you care -about. +A violation does not stop the program, but the first violation is final: +later checks, safe or not, and later `set_false_result` calls leave it in +place (SD-03). Only `map2check_init()`, `map2check_reset()` and +`vcc_reset_meta_data()` clear it. ## JSON report diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 950e0f9..6e5df78 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -17,7 +17,7 @@ differential harness in #15 will settle those cases. `tests/compat_docs.rs` checks that IDs run in order with no gaps, that every classification is from the list below, that `pending decision` and -`suspected regression` cite an issue, and that every test named under +`suspected regression` and `fixed` cite an issue, and that every test named under **Tests** exists. When a fix changes one of these behaviours, update the entry and its tests in the same change. @@ -29,6 +29,7 @@ entry and its tests in the same change. | `accepted limitation` | Known downside, accepted for now; the entry says why. | | `suspected regression` | Probably wrong; an issue tracks adjudication. | | `pending decision` | Waits on the cited issue. | +| `fixed` | Was a regression; the Rust library now matches the C runtime of Map2Check `develop`. The entry cites the fixing issue and the `develop` source. | ### SD-01 — JSON report shape @@ -76,25 +77,39 @@ map2check_init(); map2check_map_malloc((void *)0x1000, 16); set_false_result(0 /* Overflow */, 3, "main"); print_all_containers_as_json(); /* no Container_AllocationLog */ +map2check_reset(); /* the first violation is final (SD-03) */ +map2check_map_malloc((void *)0x1000, 16); set_false_result(1 /* MemsafetyFree */, 4, "main"); print_all_containers_as_json(); /* Container_AllocationLog present */ ``` -### SD-03 — Violations do not stop the analysis, and a later safe memory check clears them +### SD-03 — Violations do not stop the analysis; the first one is final -**Classification:** suspected regression (#36; differential check in #15) -**Tests:** `rust/src/ffi_tests.rs::check_load_clears_a_previous_violation_when_safe` +**Classification:** fixed (#36) +**Tests:** `rust/src/ffi_tests.rs::sd03_safe_check_keeps_earlier_violation`, `rust/src/ffi_tests.rs::sd03_later_violation_does_not_overwrite_first` **Rust:** a check that finds a violation records it and returns. The process -keeps running. The memory checks (`map2check_check_load`, -`map2check_check_free`, `map2check_check_free_resolved_address`, -`map2check_check_deref`, `map2check_check_mem_endprog`) reset the result to -OK before they check. So a later safe access erases any earlier violation, -including one that `set_false_result` recorded. - -**Legacy (embedded C library):** every violation calls `map2check_error()`, -which writes the result files and calls `abort()` -(`lib_old/Map2CheckFunctions.c:72-102`). The first violation is final. +keeps running (ending it, with the result files, is Task 2.4 of the cutover +plan). The first recorded violation is final: `AnalysisState::set_false` +ignores every later violation, whether it comes from a memory check, an +overflow check, an assert or `set_false_result`, and the memory checks +(`map2check_check_load`, `map2check_check_free`, +`map2check_check_free_resolved_address`, `map2check_check_deref`, +`map2check_check_mem_endprog`) no longer reset the result to OK before they +check. Only `map2check_init()`, `map2check_reset()` and the explicit +`vcc_reset_meta_data()` clear it. + +Before #36 those memory checks reset the result first, so a later safe +access erased any earlier violation, including one that `set_false_result` +recorded. + +**C runtime of Map2Check `develop`:** every violation calls +`map2check_error()` (`modules/backend/library/lib/Map2CheckFunctions.c:72-75`), +which calls `map2check_exit()` (`:89-102`): it writes the result files and +calls `abort()`. For example `map2check_free_resolved_address` and +`map2check_free` (`AnalysisModeMemtrack.c:173-177`, `:355-358`) and +`map2check_check_deref` (`AnalysisModeMemtrack.c:297-303`). The first +violation is therefore final. `develop` matches `lib_old` here. **Reproduce:** @@ -103,7 +118,7 @@ map2check_init(); map2check_map_malloc((void *)0x1000, 16); set_false_result(5 /* Reachability */, 1, "main"); map2check_check_load((void *)0x1000, 2, 0, 4, "main"); -/* get_current_property() == None: the REACHABILITY violation is gone */ +/* get_current_property() == Reachability (before #36: None) */ ``` ### SD-04 — End-of-program leak check is opt-in diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index 72292bb..08130a6 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -71,18 +71,6 @@ fn run_vcc_check(checker: impl VccChecker, ctx: &VccContext, line: u32, fname: & } } -fn reset_result_metadata() { - if let Err(e) = state::with_state(|s| { - s.result.ok = true; - s.result.property = ViolatedProperty::None; - s.result.line = 0; - s.result.function_name.clear(); - Ok(()) - }) { - log_error(&e); - } -} - fn violated_property_from_raw(raw: i32) -> Option { match raw { 0 => Some(ViolatedProperty::Overflow), @@ -914,7 +902,6 @@ pub unsafe extern "C" fn map2check_check_load( ) { ffi_guard( || { - reset_result_metadata(); // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; if size < 0 { @@ -957,7 +944,6 @@ pub unsafe extern "C" fn map2check_check_free( ) { ffi_guard( || { - reset_result_metadata(); // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; let ctx = VccContext::new(line, 0, fname); @@ -989,7 +975,6 @@ pub unsafe extern "C" fn map2check_check_free_resolved_address( ) { ffi_guard( || { - reset_result_metadata(); // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; if ptr.is_null() && is_null_valid != 0 { @@ -1022,7 +1007,6 @@ pub unsafe extern "C" fn map2check_check_deref( ) { ffi_guard( || { - reset_result_metadata(); // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; let ctx = VccContext::new(line, scope, fname); @@ -1045,7 +1029,6 @@ pub unsafe extern "C" fn map2check_check_deref( pub extern "C" fn map2check_check_mem_endprog() { ffi_guard( || { - reset_result_metadata(); if !state::with_state(|s| Ok(s.memcleanup_enabled)).unwrap_or(false) { return; } diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 428340c..54c1fe0 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -728,6 +728,8 @@ fn check_load_null_depends_on_null_is_valid() { unsafe { map2check_check_load(std::ptr::null(), 12, 0, 4, c"main".as_ptr()) }; assert_violated(ViolatedProperty::MemsafetyDeref, 12, "main"); + // The first violation is final (SD-03), so start a new analysis. + state::reset(); map2check_set_null_is_valid(); // SAFETY: as above. unsafe { map2check_check_load(std::ptr::null(), 12, 0, 4, c"main".as_ptr()) }; @@ -735,15 +737,46 @@ fn check_load_null_depends_on_null_is_valid() { } #[test] -fn check_load_clears_a_previous_violation_when_safe() { +fn sd03_safe_check_keeps_earlier_violation() { + // SD-03 / #36: the first violation is final. A later memory check that + // finds the access safe must not reset the result to OK. let _guard = fresh(); // SAFETY: C-string literals are valid; fake addresses are never dereferenced. unsafe { map2check_map_malloc(addr(0x1000), 16); - set_false_result(0, 1, c"earlier".as_ptr()); - map2check_check_load(addr(0x1000), 12, 0, 4, c"main".as_ptr()); + set_false_result(5, 1, c"main".as_ptr()); + map2check_check_load(addr(0x1000), 2, 0, 4, c"main".as_ptr()); } - assert_ok(); + assert_violated(ViolatedProperty::Reachability, 1, "main"); + assert_eq!(get_current_property(), ViolatedProperty::Reachability); + + // SAFETY: as above. + unsafe { + map2check_check_deref(addr(0x1000), 0, 3, c"main".as_ptr()); + map2check_check_free(c"p".as_ptr(), addr(0x1000), 0, 4, c"main".as_ptr()); + map2check_check_free_resolved_address(addr(0x1000), 5, c"main".as_ptr(), 0); + map2check_map_free(c"p".as_ptr(), addr(0x1000), 0, 5, c"main".as_ptr()); + } + map2check_check_mem_endprog(); + map2check_set_memcleanup(); + map2check_check_mem_endprog(); + assert_violated(ViolatedProperty::Reachability, 1, "main"); +} + +#[test] +fn sd03_later_violation_does_not_overwrite_first() { + // SD-03 / #36: neither a later memory violation nor a later + // `set_false_result` replaces the recorded one. + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_check_load(addr(0x9000), 7, 0, 4, c"first".as_ptr()); + map2check_check_free(c"p".as_ptr(), addr(0x9000), 0, 8, c"main".as_ptr()); + map2check_check_deref(std::ptr::null(), 0, 9, c"main".as_ptr()); + set_false_result(5, 10, c"main".as_ptr()); + map2check_binop_add_int(i32::MAX, 1, 11, 0, c"main".as_ptr()); + } + assert_violated(ViolatedProperty::MemsafetyDeref, 7, "first"); } #[test] @@ -820,10 +853,13 @@ fn check_deref_accepts_live_and_rejects_freed_null_or_untracked() { } assert_violated(ViolatedProperty::MemsafetyDeref, 42, "main"); + // The first violation is final (SD-03), so each case starts a new analysis. + state::reset(); // SAFETY: as above. unsafe { map2check_check_deref(std::ptr::null(), 0, 43, c"main".as_ptr()) }; assert_violated(ViolatedProperty::MemsafetyDeref, 43, "main"); + state::reset(); // SAFETY: as above. unsafe { map2check_check_deref(addr(0x9000), 0, 44, c"main".as_ptr()) }; assert_violated(ViolatedProperty::MemsafetyDeref, 44, "main"); diff --git a/rust/src/state.rs b/rust/src/state.rs index 79a80d9..3689005 100644 --- a/rust/src/state.rs +++ b/rust/src/state.rs @@ -33,7 +33,16 @@ impl AnalysisState { *self = Self::default(); } + /// Records a violation unless one is already recorded (#36, SD-03). + /// + /// The first violation is final: in the C runtime of Map2Check every + /// violation ends the process through `map2check_error()`, so nothing can + /// replace it. This library keeps running instead, so a later violation is + /// ignored rather than overwriting the first. pub fn set_false(&mut self, property: ViolatedProperty, line: u32, function_name: &str) { + if !self.result.ok { + return; + } self.result.ok = false; self.result.property = property; self.result.line = line; diff --git a/tests/compat_docs.rs b/tests/compat_docs.rs index b69074b..350356b 100644 --- a/tests/compat_docs.rs +++ b/tests/compat_docs.rs @@ -321,6 +321,7 @@ const CLASSIFICATIONS: &[&str] = &[ "accepted limitation", "suspected regression", "pending decision", + "fixed", ]; /// Problemas de uma entrada: classificação conhecida (com issue quando @@ -337,7 +338,7 @@ fn sd_entry_problems(id: &str, body: &str, root: &Path) -> Vec { Some(c) => match CLASSIFICATIONS.iter().find(|k| c.starts_with(*k)) { None => problems.push(format!("{id}: classificação desconhecida `{c}`")), Some(k) - if matches!(*k, "suspected regression" | "pending decision") + if matches!(*k, "suspected regression" | "pending decision" | "fixed") && !c.contains('#') => { problems.push(format!("{id}: `{k}` precisa citar uma issue (#N)")) diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index 71c601f..13f9942 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -332,10 +332,13 @@ static int test_result_and_control_flags(void) { return 4; } + /* SD-03 (#36): the first violation is final; a later safe check and a + * later violation leave it in place. */ map2check_reset(); set_false_result(Overflow, 10, "main"); map2check_check_free(NULL, NULL, 0, 11, NULL); - if (get_current_property() != None) { + set_false_result(Reachability, 12, "main"); + if (get_current_property() != Overflow) { return 5; } return 0; From 315e3e3c38c3f11e5b49df45ead187dc8244a466 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 20:53:02 -0400 Subject: [PATCH 13/63] fix: report an untracked resolved free as invalid (#37) map2check_check_free_resolved_address treated an address with no heap record as a safe free, while map2check_check_free reported MEMSAFETY_FREE for it. The C runtime of Map2Check develop uses is_invalid_free (ListLog.c:95-118) on both free paths, and it returns true when no row points to the address. is_invalid_free_resolved now lets the most recent heap record (allocation or free) decide and reports an address with none as invalid. Invert the pinning tests, add sd06_untracked_free_is_invalid and a stack-slot case, mark SD-06 fixed and update api-deltas and the ABI mapping row. --- docs/api-deltas.md | 16 +++++++-------- docs/cutover/abi-mapping.tsv | 2 +- docs/semantic-deltas.md | 35 ++++++++++++++++++++++----------- rust/src/analysismode/memory.rs | 3 ++- rust/src/ffi_tests.rs | 28 ++++++++++++++++++++++++++ rust/src/memtrack/mod.rs | 31 ++++++++++++++++++++++++----- 6 files changed, 89 insertions(+), 26 deletions(-) diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 9c409b0..45e72af 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -84,7 +84,7 @@ build with `--features libfuzzer`. | `map2check_binop_shr_unsigned` | `void map2check_binop_shr_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shr_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | | `map2check_check_load` | `void map2check_check_load(const void *ptr, int32_t line, uint32_t scope, int32_t size, const char *function_name)` | `map2check_check_load` | changed | Address-validity rule rewritten (plan R1; differential check in #15). The first violation is final; a later check never clears or replaces it (SD-03); NULL handling in SD-05. | | `map2check_check_free` | `void map2check_check_free(const char *_name, const void *ptr, uint32_t _scope, uint32_t line, const char *function_name)` | `map2check_check_free` | same | `_name` and `_scope` are ignored. Untracked address is a violation (SD-06). | -| `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; untracked address is safe (SD-06). | +| `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; untracked address is a violation (SD-06). | | `map2check_check_deref` | `void map2check_check_deref(const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_deref` | same | Name collides with the pass ABI `map2check_check_deref(i64, ptr)`; see the pass matrix (#21). | | `map2check_check_mem_endprog` | `void map2check_check_mem_endprog(void)` | `map2check_check_mem_endprog` | changed | Leak algorithm rewritten; no-op until `map2check_set_memcleanup()` (SD-04). | | `LLVMFuzzerTestOneInput` | `int32_t LLVMFuzzerTestOneInput(const uint8_t *data, uintptr_t size)` | `LLVMFuzzerTestOneInput` | same | Feature `libfuzzer`. Resets the analysis state on every iteration (SD-11). | @@ -264,13 +264,13 @@ receives. The Rust export: - returns without a check when `ptr` is NULL and `is_null_valid != 0`; - reports `MEMSAFETY_FREE` when the most recent record whose `mem_address_points_to` equals `ptr` is already freed; -- treats an address with no record as safe; +- reports `MEMSAFETY_FREE` for an address with no heap record (SD-06, #37); - only checks: it does not mark the block freed. The caller records the release with `map2check_map_free`. -The embedded C library of Map2Check -(`lib_old/AnalysisModeMemtrack.c:164-183` and `lib_old/ListLog.c:95-118` on -`origin/refactory_memory_log`) differs in two ways: an address with no -record is an invalid free, and the function also marks the block freed. The -first difference is SD-06. The second keeps the split between checking and -recording that the rest of the core API already uses. +The C runtime of Map2Check `develop` +(`modules/backend/library/lib/AnalysisModeMemtrack.c:164-183` and +`ListLog.c:95-118`) agrees on the untracked address since #37, and still +differs in one way: the function also marks the block freed. That keeps the +split between checking and recording that the rest of the core API already +uses. diff --git a/docs/cutover/abi-mapping.tsv b/docs/cutover/abi-mapping.tsv index a4b324a..057fb3c 100644 --- a/docs/cutover/abi-mapping.tsv +++ b/docs/cutover/abi-mapping.tsv @@ -24,7 +24,7 @@ map2check_add_store_pointer map2check_map_store_pointer - C: (ptr var,ptr value, map2check_load map2check_check_load - C: (ptr,i64 size) (MemoryTrackPass.cpp:770) → núcleo: (ptr,line:i32,scope:u32,size:i32,fn); o passe passa a emitir linha, escopo e função (DebugInfo); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (a flag ERROR_DEREF de AnalysisModeMemtrack.c:158 só encerra em check_deref); check_load grava a violação direto map2check_check_deref - - check_load já registra a violação (ffi.rs:894, LoadChecker); C: (i64 line,ptr fn) (MemoryTrackPass.cpp:785) só lê a flag ERROR_DEREF e encerra (AnalysisModeMemtrack.c:297); o passe deixa de emitir; C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) map2check_free map2check_check_free+map2check_map_free - C: (ptr name,ptr var_slot,i64 scope,i64 line,ptr fn) (MemoryTrackPass.cpp:818) → núcleo (ambos): (name,ptr,scope:u32,line:u32,fn); semântica confirmada: o núcleo é chaveado no endereço de heap carregado (ffi_tests.rs:722-744), então o passe carrega *ptr antes da chamada e a ordem é check_free depois map_free; o passe não chama nenhum dos dois se o valor carregado for NULL, como o C (AnalysisModeMemtrack.c:342); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:352) -map2check_free_resolved_address map2check_check_free_resolved_address - SD-06 (#37); C: (ptr,i64 line,ptr fn,i1 nullValid) (MemoryTrackPass.cpp:774) → núcleo: (ptr,line:u32,fn,is_null_valid:i16); i1 passa a i16 (zext); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:170) +map2check_free_resolved_address map2check_check_free_resolved_address - SD-06 (#37, corrigido): endereço sem registro reporta MEMSAFETY_FREE, como is_invalid_free (ListLog.c:95-118); C: (ptr,i64 line,ptr fn,i1 nullValid) (MemoryTrackPass.cpp:774) → núcleo: (ptr,line:u32,fn,is_null_valid:i16); i1 passa a i16 (zext); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:170) map2check_posix map2check_map_posix_memalign 2.5 chamado em MemoryTrackPass.cpp:120 para posix_memalign; não é obsoleto; C: (ptr,i64 size) (MemoryTrackPass.cpp:793); o C runtime desreferencia ptr (void**) e registra malloc+store_pointer (AnalysisModeMemtrack.c:328); núcleo proposto: (ptr_slot,size:i32) map2check_wasm_malloc map2check_wasm_malloc 2.8 C: (i64,i64) (MemoryTrackPass.cpp:825) map2check_wasm_free map2check_wasm_free 2.8 C: (i64,i64) (MemoryTrackPass.cpp:829) diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 6e5df78..b089e24 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -168,22 +168,35 @@ map2check_check_load(NULL, 12, 0, 4, "main"); /* None */ ### SD-06 — Freeing an address with no record -**Classification:** suspected regression (#37; differential check in #15) -**Tests:** `rust/src/ffi_tests.rs::check_free_rejects_double_free_and_untracked_address`, `rust/src/analysismode/memory.rs::free_resolved_checker_outcomes` - -**Rust:** `map2check_check_free` reports `MEMSAFETY_FREE` for an address -with no record, but `map2check_check_free_resolved_address` treats that -address as safe. - -**Legacy (embedded C library):** `is_invalid_free` returns true for an -address with no record (`lib_old/ListLog.c:95-118`), and both -`map2check_free` and `map2check_free_resolved_address` use it. +**Classification:** fixed (#37) +**Tests:** `rust/src/ffi_tests.rs::sd06_untracked_free_is_invalid`, `rust/src/ffi_tests.rs::check_free_resolved_address_rejects_stack_slot`, `rust/src/ffi_tests.rs::check_free_rejects_double_free_and_untracked_address`, `rust/src/analysismode/memory.rs::free_resolved_checker_outcomes`, `rust/src/memtrack/mod.rs::free_resolved_unknown_address_is_invalid` + +**Rust:** `map2check_check_free` and `map2check_check_free_resolved_address` +both report `MEMSAFETY_FREE` for an address with no heap record (no +allocation or free event; a stack slot or a pointer store does not count). +The most recent heap record for the address decides: a freed block is an +invalid free, a live one is valid. + +Before #37, `map2check_check_free_resolved_address` treated an address with +no record as a safe free (`is_invalid_free_resolved` ended in +`.unwrap_or(false)`). + +**C runtime of Map2Check `develop`:** `is_invalid_free` +(`modules/backend/library/lib/ListLog.c:95-118`) returns false for NULL, +lets the most recent `list_log` row pointing to the address decide +(invalid if freed or not dynamic) and returns true when no row is found. +Both `map2check_free` (`AnalysisModeMemtrack.c:351`) and +`map2check_free_resolved_address` (`AnalysisModeMemtrack.c:171`) use it in +memtrack mode. `develop` matches `lib_old` here. (In memcleanup mode, +`AnalysisModeMemcleanup.c:167` and `:341` hard-code `error = FALSE`; this +library does not have that mode split.) **Reproduce:** ```c map2check_init(); -map2check_check_free_resolved_address((void *)0x5000, 3, "main", 0); /* None */ +map2check_check_free_resolved_address((void *)0x5000, 3, "main", 0); /* MemsafetyFree (before #37: None) */ +map2check_reset(); map2check_check_free("p", (void *)0x5000, 0, 4, "main"); /* MemsafetyFree */ ``` diff --git a/rust/src/analysismode/memory.rs b/rust/src/analysismode/memory.rs index e0d4eeb..4b85aaa 100644 --- a/rust/src/analysismode/memory.rs +++ b/rust/src/analysismode/memory.rs @@ -295,11 +295,12 @@ mod tests { .unwrap(), violated(ViolatedProperty::MemsafetyFree) ); + // SD-06 / #37: an untracked address is an invalid free. assert_eq!( FreeResolvedChecker { address: 0x9999 } .check(&ctx()) .unwrap(), - VccOutcome::Safe + violated(ViolatedProperty::MemsafetyFree) ); } diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 54c1fe0..897e3cf 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -828,6 +828,34 @@ fn check_free_resolved_address_rejects_freed_target() { assert_violated(ViolatedProperty::MemsafetyFree, 31, "main"); } +#[test] +fn sd06_untracked_free_is_invalid() { + // SD-06 / #37: both free paths use develop's `is_invalid_free`, which + // reports an address with no record as an invalid free. + let _guard = fresh(); + // SAFETY: C-string literal is valid; fake address is never dereferenced. + unsafe { map2check_check_free_resolved_address(addr(0x5000), 3, c"main".as_ptr(), 0) }; + assert_violated(ViolatedProperty::MemsafetyFree, 3, "main"); + + state::reset(); + // SAFETY: as above. + unsafe { map2check_check_free(c"p".as_ptr(), addr(0x5000), 0, 4, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::MemsafetyFree, 4, "main"); +} + +#[test] +fn check_free_resolved_address_rejects_stack_slot() { + // develop: a stack slot never reaches list_log as a dynamic row, so + // `is_invalid_free` reports it (ListLog.c:95-118). + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake address is never dereferenced. + unsafe { + map2check_map_alloca(c"x".as_ptr(), addr(0x6000), 8, 4, 1, 0); + map2check_check_free_resolved_address(addr(0x6000), 5, c"main".as_ptr(), 0); + } + assert_violated(ViolatedProperty::MemsafetyFree, 5, "main"); +} + #[test] fn check_free_resolved_address_null_depends_on_flag() { let _guard = fresh(); diff --git a/rust/src/memtrack/mod.rs b/rust/src/memtrack/mod.rs index 8885c4c..3228aba 100644 --- a/rust/src/memtrack/mod.rs +++ b/rust/src/memtrack/mod.rs @@ -110,13 +110,17 @@ pub fn is_invalid_free(entries: &[MemTrackEntry], address: usize) -> bool { /// the value obtained after following one level of indirection. Used by /// `map2check_check_free_resolved_address`, which receives the resolved /// pointer value directly instead of the address of the pointer variable. +/// +/// Follows `is_invalid_free` of the Map2Check `develop` C runtime +/// (`ListLog.c:95-118`): the most recent heap record (allocation or free) for +/// `address` decides, a freed block is invalid, and an address with no heap +/// record — untracked, or only a stack slot — is an invalid free (#37). pub fn is_invalid_free_resolved(entries: &[MemTrackEntry], address: usize) -> bool { entries .iter() .rev() - .find(|e| e.points_to == address) - .map(|e| e.is_free) - .unwrap_or(false) + .find(|e| (e.is_dynamic || e.is_free) && e.points_to == address) + .is_none_or(|e| e.is_free) } /// Checks whether dereferencing `address` is invalid (not allocated or already freed). @@ -211,9 +215,26 @@ mod tests { } #[test] - fn free_resolved_unknown_address_not_flagged() { + fn free_resolved_unknown_address_is_invalid() { let entries = vec![make_entry(0x100, 0x200, true, false, 8)]; - assert!(!is_invalid_free_resolved(&entries, 0xDEAD)); + assert!(is_invalid_free_resolved(&entries, 0xDEAD)); + } + + #[test] + fn free_resolved_stack_slot_is_invalid() { + let entries = vec![make_entry(0x300, 0x300, false, false, 8)]; + assert!(is_invalid_free_resolved(&entries, 0x300)); + } + + #[test] + fn free_resolved_ignores_pointer_stores_to_the_block() { + // A later `store_pointer` record (`p = block`) is not an allocation + // event and must not hide the live allocation. + let entries = vec![ + make_entry(0x200, 0x200, true, false, 8), + make_entry(0x500, 0x200, false, false, 0), + ]; + assert!(!is_invalid_free_resolved(&entries, 0x200)); } #[test] From b5609464fe039582aa9bebb7289686799a250c1f Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 20:54:43 -0400 Subject: [PATCH 14/63] fix: register a VLA as a static slot sized in elements (#38) map2check_map_non_static_alloca marked its record dynamic, so a VLA was reported as MEMSAFETY_MEMCLEANUP at end of program, and it took size in bytes. The C runtime of Map2Check develop forwards to map2check_alloca with size * size_of_primitive (AnalysisModeMemtrack.c:368-372), a static heap_log slot that the leak check never sees. The export now registers a static slot of size * size_primitive bytes and drops the record when that product overflows. The signature is unchanged. Replace the three pinning tests with sd12_vla_is_not_a_leak_and_size_is_elements and companions, mark SD-12 fixed and update api-deltas and the ABI mapping row. --- docs/api-deltas.md | 20 ++++++------ docs/cutover/abi-mapping.tsv | 2 +- docs/semantic-deltas.md | 54 ++++++++++++++++----------------- rust/src/ffi.rs | 20 +++++++----- rust/src/ffi_tests.rs | 59 +++++++++++++++++++++++------------- 5 files changed, 87 insertions(+), 68 deletions(-) diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 45e72af..b686447 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -63,7 +63,7 @@ build with `--features libfuzzer`. | `map2check_save_basic_block_log` | `bbtrack_log_t *map2check_save_basic_block_log(int32_t line, const char *function_name)` | returns `bbtrack_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_is_in_trackbb_container` | `int32_t map2check_is_in_trackbb_container(int32_t line)` | `map2check_is_in_trackbb_container` | same | | | `map2check_map_alloca` | `void map2check_map_alloca(const char *var_name, const void *ptr_address, int32_t size, int32_t size_primitive, int32_t line_number, int32_t scope)` | `container_memtracklog.h:41` | same | | -| `map2check_map_non_static_alloca` | `void map2check_map_non_static_alloca(const char *var_name, const void *ptr_address, int32_t size, int32_t size_primitive, int32_t line_number, int32_t scope)` | `container_memtracklog.h:44` | same | Signature and semantics reconstructed; see its section. Counted as a leak at end of program (SD-12). | +| `map2check_map_non_static_alloca` | `void map2check_map_non_static_alloca(const char *var_name, const void *ptr_address, int32_t size, int32_t size_primitive, int32_t line_number, int32_t scope)` | `container_memtracklog.h:44` | same | `size` is an element count; static slot of `size * size_primitive` bytes, never a leak (SD-12, #38). | | `map2check_map_funct_address` | `void map2check_map_funct_address(const char *var_name, const void *ptr_address)` | `container_memtracklog.h:48` | same | Signature and semantics reconstructed; see its section. | | `map2check_map_malloc` | `void map2check_map_malloc(const void *ptr_address, int32_t size)` | `map2check_map_malloc` | same | Negative sizes are ignored (SD-09). | | `map2check_map_calloc` | `void map2check_map_calloc(const void *ptr_address, int32_t quantity, int32_t size)` | `map2check_map_calloc` | same | Negative sizes are ignored (SD-09). | @@ -216,21 +216,19 @@ semantics were reconstructed from the paridade table and from the existing `map2check_map_alloca`/`map2check_map_malloc` patterns in `rust/src/ffi.rs`, not from the original C source: -- `map2check_map_non_static_alloca` — same shape as `map2check_map_alloca`, - but the resulting entry is marked dynamic (`entry.set_malloc()`), so - `map2check_check_mem_endprog` accounts for it like a heap allocation - instead of a plain stack slot. +- `map2check_map_non_static_alloca` — same shape as `map2check_map_alloca`. + The M3 reconstruction marked the entry dynamic and took `size` in bytes; + #38 replaced that with the `develop` semantics below. - `map2check_map_funct_address` — registers a function's address as an always-valid load target (pointer-width `size_destiny`), so that reading a function pointer through `map2check_check_load`/`map2check_check_deref` does not get flagged as an invalid address. -The embedded C library of Map2Check (`lib_old/AnalysisModeMemtrack.c:366-377` -on `origin/refactory_memory_log`) does the opposite for -`map2check_non_static_alloca`: it registers a static slot of -`size * size_of_primitive` bytes, so its `size` is an element count. The -Rust export takes `size` in bytes, like `map2check_map_alloca`, and its -choice makes an unfreed VLA show up as a leak at end of program; see SD-12. +Since #38 `map2check_map_non_static_alloca` follows the C runtime of +Map2Check `develop` (`modules/backend/library/lib/AnalysisModeMemtrack.c:368-372`): +`size` is an element count and the export registers a static slot of +`size * size_primitive` bytes, so a VLA never shows up as a leak at end of +program; see SD-12. **Risk:** if the M5 compatibility layer (`ffi_compat.rs`, consuming the exact signatures the Map2Check LLVM passes insert) reveals a different expected diff --git a/docs/cutover/abi-mapping.tsv b/docs/cutover/abi-mapping.tsv index 057fb3c..4b0617e 100644 --- a/docs/cutover/abi-mapping.tsv +++ b/docs/cutover/abi-mapping.tsv @@ -18,7 +18,7 @@ map2check_binop_srem map2check_binop_rem_signed 2.5 C: (i64 a,i64 b,i32 width,i3 map2check_malloc map2check_map_malloc - C: (ptr,i64 size) (MemoryTrackPass.cpp:789) → núcleo: (ptr,size:i32); size estreita de i64 para i32 map2check_calloc map2check_map_calloc - C: (ptr,i64 qty,i64 size) → núcleo: (ptr,qty:i32,size:i32); mesma ordem, estreita para i32 map2check_alloca map2check_map_alloca - C: (ptr name,ptr addr,i64 size,i64 size_primitive,i64 line,i64 scope) (MemoryTrackPass.cpp:802) → núcleo: (name,addr,size:i32,size_primitive:i32,line:i32,scope:i32); mesma ordem, estreita para i32 -map2check_non_static_alloca map2check_map_non_static_alloca - SD-12 (#38): unidade de tamanho; assinatura igual a map2check_alloca (MemoryTrackPass.cpp:808); C multiplica size*size_primitive (AnalysisModeMemtrack.c:370) +map2check_non_static_alloca map2check_map_non_static_alloca - SD-12 (#38, corrigido): o núcleo agora toma size como contagem de elementos e registra slot estático de size*size_primitive bytes (nunca vazamento), como o C; assinatura do núcleo mantida, igual a map2check_alloca (MemoryTrackPass.cpp:808); C multiplica size*size_primitive (AnalysisModeMemtrack.c:370) map2check_function map2check_map_funct_address - C: (ptr name,ptr addr) (MemoryTrackPass.cpp:814) → núcleo: (var_name,ptr_address); igual map2check_add_store_pointer map2check_map_store_pointer - C: (ptr var,ptr value,i64 scope,ptr name,i64 line,ptr fn) (MemoryTrackPass.cpp:779) → núcleo: (var_address,value,scope:u32,var_name,line:i32,fn); mesma ordem map2check_load map2check_check_load - C: (ptr,i64 size) (MemoryTrackPass.cpp:770) → núcleo: (ptr,line:i32,scope:u32,size:i32,fn); o passe passa a emitir linha, escopo e função (DebugInfo); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (a flag ERROR_DEREF de AnalysisModeMemtrack.c:158 só encerra em check_deref); check_load grava a violação direto diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index b089e24..ce3ff26 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -299,38 +299,38 @@ the next. **Reproduce:** run any LibFuzzer target linked with `--features libfuzzer` for two inputs; the second report has no records from the first. -### SD-12 — `map2check_map_non_static_alloca`: a VLA counts as a leak, and `size` is in bytes - -**Classification:** suspected regression (#38; differential check in #15) -**Tests:** `rust/src/ffi_tests.rs::map_non_static_alloca_records_dynamic_slot`, `rust/src/ffi_tests.rs::non_static_alloca_is_reported_as_leak_at_end_of_program`, `rust/src/ffi_tests.rs::non_static_alloca_size_is_bytes_not_element_count` - -**Rust:** `map2check_map_non_static_alloca` differs from the legacy call in -two ways: - -- It marks its record dynamic, as the reconstructed semantics in - `api-deltas.md` describe. With `map2check_set_memcleanup()` on, a VLA - that goes out of scope without a matching free reports - `MEMSAFETY_MEMCLEANUP`. -- It stores `size` as the allocation size in bytes, like - `map2check_map_alloca`. A caller that passes the element count, as the - legacy contract expects, registers a block that is too small. Loads past - the first `size` bytes then report `MEMSAFETY_DEREF`. - -**Legacy (embedded C library):** `map2check_non_static_alloca` treats -`size` as an element count and registers a static slot of -`size * size_of_primitive` bytes (`lib_old/AnalysisModeMemtrack.c:366-377`). -So a VLA never counts as a leak. +### SD-12 — `map2check_map_non_static_alloca`: a VLA is a static slot, and `size` is an element count + +**Classification:** fixed (#38) +**Tests:** `rust/src/ffi_tests.rs::sd12_vla_is_not_a_leak_and_size_is_elements`, `rust/src/ffi_tests.rs::map_non_static_alloca_records_static_slot_of_elements_times_primitive`, `rust/src/ffi_tests.rs::map_non_static_alloca_ignores_negative_or_overflowing_size` + +**Rust:** `map2check_map_non_static_alloca(name, ptr, size, size_primitive, +line, scope)` registers a static slot of `size * size_primitive` bytes, +like `map2check_map_alloca` with that size. A VLA never counts as a leak at +end of program, and loads anywhere inside its `size` elements are valid. A +negative argument, or a byte size that overflows `usize`, drops the record +(SD-09). The signature is unchanged. + +Before #38 the record was marked dynamic (a VLA reported +`MEMSAFETY_MEMCLEANUP` with `map2check_set_memcleanup()` on) and `size` was +taken in bytes, so a caller passing the element count registered a block +that was too small. + +**C runtime of Map2Check `develop`:** `map2check_non_static_alloca` +(`modules/backend/library/lib/AnalysisModeMemtrack.c:368-372`) calls +`map2check_alloca(name, ptr, size * size_of_primitive, size_of_primitive, +line, scope)`, which appends a `heap_log` row (`:374-379`). `heap_log` rows +are stack/static slots: `map2check_load` accepts them +(`HeapLog.c:37-52`) and the leak check only walks `allocation_log` +(`AllocationLog.c:69-101`). `develop` matches `lib_old` here. **Reproduce:** ```c map2check_init(); int n = 4; int vla[n]; -map2check_map_non_static_alloca("vla", vla, sizeof(vla), sizeof(int), 3, 0); +map2check_map_non_static_alloca("vla", vla, n, sizeof(int), 3, 0); /* element count */ map2check_set_memcleanup(); -map2check_check_mem_endprog(); /* MemsafetyMemcleanup */ - -map2check_reset(); -map2check_map_non_static_alloca("vla", vla, n, sizeof(int), 3, 0); /* legacy: element count */ -map2check_check_load(&vla[2], 4, 0, sizeof(int), "main"); /* MemsafetyDeref; legacy: valid */ +map2check_check_mem_endprog(); /* None (before #38: MemsafetyMemcleanup) */ +map2check_check_load(&vla[2], 4, 0, sizeof(int), "main"); /* None (before #38: MemsafetyDeref) */ ``` diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index 08130a6..ad94837 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -497,9 +497,12 @@ pub unsafe extern "C" fn map2check_map_alloca( } /// Registers a non-static (dynamically-sized) stack allocation, e.g. a -/// variable-length array whose size is only known at runtime. Unlike -/// `map2check_map_alloca`, the resulting entry is marked dynamic so that -/// `map2check_check_mem_endprog` accounts for it like a heap allocation. +/// variable-length array whose size is only known at runtime. `size` is the +/// element count: the slot covers `size * size_primitive` bytes. Like +/// `map2check_map_alloca`, the slot is static, so it is never a leak at end of +/// program. This matches `map2check_non_static_alloca` of the Map2Check +/// `develop` C runtime (`AnalysisModeMemtrack.c:368-372`; #38, SD-12). A +/// negative argument or a byte size that overflows drops the record (SD-09). /// /// # Safety /// `var_name` must be null or a valid null-terminated C string; `ptr_address` may be null. @@ -517,12 +520,15 @@ pub unsafe extern "C" fn map2check_map_non_static_alloca( if size < 0 || size_primitive < 0 { return; } + let Some(bytes) = (size as usize).checked_mul(size_primitive as usize) else { + return; + }; // SAFETY: caller guarantees var_name is a valid C string or null. let name = unsafe { c_str(var_name) }; let addr = ptr_address as usize; if let Err(e) = state::with_state(|s| { let step = s.next_step(); - let mut entry = MemTrackEntry::new( + s.memtrack.push(MemTrackEntry::new( step, line_number as u32, scope as u32, @@ -532,12 +538,10 @@ pub unsafe extern "C" fn map2check_map_non_static_alloca( false, name, "", - size as usize, + bytes, size_primitive as usize, false, - ); - entry.set_malloc(); - s.memtrack.push(entry); + )); Ok(()) }) { log_error(&e); diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 897e3cf..52063b0 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -388,43 +388,60 @@ fn map_alloca_ignores_negative_size() { } #[test] -fn map_non_static_alloca_records_dynamic_slot() { +fn map_non_static_alloca_records_static_slot_of_elements_times_primitive() { + // develop: `map2check_non_static_alloca` calls `map2check_alloca` with + // `size * size_of_primitive` (AnalysisModeMemtrack.c:368-372). let _guard = fresh(); // SAFETY: C-string literal is valid; fake address is never dereferenced. - unsafe { map2check_map_non_static_alloca(c"vla".as_ptr(), addr(0x2000), 40, 4, 5, 2) }; + unsafe { map2check_map_non_static_alloca(c"vla".as_ptr(), addr(0x2000), 10, 4, 5, 2) }; let e = only_memtrack(); - assert_eq!((e.line, e.scope, e.size_destiny), (5, 2, 40)); + assert_eq!((e.line, e.scope), (5, 2)); + assert_eq!((e.size_destiny, e.size_primitive), (40, 4)); + assert_eq!((e.var_addr, e.points_to), (0x2000, 0x2000)); assert_eq!(e.ptr_name, "vla"); - assert!(e.is_dynamic && !e.is_free); + assert!(!e.is_dynamic && !e.is_free); } #[test] -fn non_static_alloca_is_reported_as_leak_at_end_of_program() { - // Fixa o comportamento atual descrito em SD-12 (docs/semantic-deltas.md); - // o legado registra o VLA como slot estático e não acusa vazamento. +fn sd12_vla_is_not_a_leak_and_size_is_elements() { + // SD-12 / #38: a VLA is a static slot (never a leak) and `size` is an + // element count multiplied by `size_primitive`. let _guard = fresh(); - // SAFETY: C-string literal is valid; fake address is never dereferenced. - unsafe { map2check_map_non_static_alloca(c"vla".as_ptr(), addr(0x2000), 16, 4, 3, 0) }; + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { map2check_map_non_static_alloca(c"vla".as_ptr(), addr(0x2000), 4, 4, 3, 0) }; map2check_set_memcleanup(); map2check_check_mem_endprog(); - assert_violated(ViolatedProperty::MemsafetyMemcleanup, 0, "end_of_program"); -} + assert_ok(); -#[test] -fn non_static_alloca_size_is_bytes_not_element_count() { - // Fixa SD-12: o legado multiplica `size` por `size_primitive`; aqui `size` - // já é o tamanho em bytes, então passar a contagem de elementos (4 ints) - // registra só 4 bytes e o acesso a vla[2] é acusado. - let _guard = fresh(); - // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + state::reset(); + // SAFETY: as above. unsafe { map2check_map_non_static_alloca(c"vla".as_ptr(), addr(0x2000), 4, 4, 3, 0); - map2check_check_load(addr(0x2000), 4, 0, 4, c"main".as_ptr()); + map2check_check_load(addr(0x2008), 4, 0, 4, c"main".as_ptr()); + map2check_check_load(addr(0x200c), 4, 0, 4, c"main".as_ptr()); } assert_ok(); // SAFETY: as above. - unsafe { map2check_check_load(addr(0x2008), 4, 0, 4, c"main".as_ptr()) }; - assert_violated(ViolatedProperty::MemsafetyDeref, 4, "main"); + unsafe { map2check_check_load(addr(0x2010), 5, 0, 4, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::MemsafetyDeref, 5, "main"); +} + +#[test] +fn map_non_static_alloca_ignores_negative_or_overflowing_size() { + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake address is never dereferenced. + unsafe { + map2check_map_non_static_alloca(c"vla".as_ptr(), addr(0x2000), -1, 4, 3, 0); + map2check_map_non_static_alloca(c"vla".as_ptr(), addr(0x2000), 4, -4, 3, 0); + } + assert!(snapshot(|s| s.memtrack.is_empty())); + if usize::BITS == 32 { + // SAFETY: as above. + unsafe { + map2check_map_non_static_alloca(c"vla".as_ptr(), addr(0x2000), i32::MAX, 4, 3, 0) + }; + assert!(snapshot(|s| s.memtrack.is_empty())); + } } #[test] From 1ff05665d54fccfb91dc5b2368df71742917115d Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 20:56:18 -0400 Subject: [PATCH 15/63] fix: validate addresses by the latest alloc/free event (#11) map2check_check_load accepted an access to a freed block because the older malloc record still covered the address. In the C runtime of Map2Check develop, the allocation log keeps one row per address, updated in place on malloc and free (AnalysisModeMemtrack.c:313-317, :360-363), so only the latest event counts (AllocationLog.c:127-148). is_invalid_address now lets the latest heap event per block address decide, keeps stack slots valid and ignores pointer stores. Add tests for use-after-free (load and deref), reallocation at the same address, a smaller reallocation, range boundaries and a pointer store after free. --- docs/api-deltas.md | 2 +- rust/src/ffi_tests.rs | 113 +++++++++++++++++++++++++++++++++++++++ rust/src/memtrack/mod.rs | 52 ++++++++++++++++-- 3 files changed, 162 insertions(+), 5 deletions(-) diff --git a/docs/api-deltas.md b/docs/api-deltas.md index b686447..21603a0 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -82,7 +82,7 @@ build with `--features libfuzzer`. | `map2check_binop_mul_unsigned` | `void map2check_binop_mul_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_mul_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | | `map2check_binop_div_unsigned` | `void map2check_binop_div_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_div_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | | `map2check_binop_shr_unsigned` | `void map2check_binop_shr_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shr_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_check_load` | `void map2check_check_load(const void *ptr, int32_t line, uint32_t scope, int32_t size, const char *function_name)` | `map2check_check_load` | changed | Address-validity rule rewritten (plan R1; differential check in #15). The first violation is final; a later check never clears or replaces it (SD-03); NULL handling in SD-05. | +| `map2check_check_load` | `void map2check_check_load(const void *ptr, int32_t line, uint32_t scope, int32_t size, const char *function_name)` | `map2check_check_load` | changed | Address-validity rule rewritten (plan R1; differential check in #15); since #11 a heap block is valid only while its latest alloc/free event is an allocation, as in `develop` (`AllocationLog.c:127-148`). The first violation is final; a later check never clears or replaces it (SD-03); NULL handling in SD-05. | | `map2check_check_free` | `void map2check_check_free(const char *_name, const void *ptr, uint32_t _scope, uint32_t line, const char *function_name)` | `map2check_check_free` | same | `_name` and `_scope` are ignored. Untracked address is a violation (SD-06). | | `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; untracked address is a violation (SD-06). | | `map2check_check_deref` | `void map2check_check_deref(const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_deref` | same | Name collides with the pass ABI `map2check_check_deref(i64, ptr)`; see the pass matrix (#21). | diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 52063b0..13c69c1 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -796,6 +796,119 @@ fn sd03_later_violation_does_not_overwrite_first() { assert_violated(ViolatedProperty::MemsafetyDeref, 7, "first"); } +// ── #11: validity follows the latest alloc/free event ─────────────────────── + +#[test] +fn deref_after_free_is_invalid() { + let _guard = fresh(); + map2check_init(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x2000), 16); + map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_check_deref(addr(0x2000), 0, 10, c"main".as_ptr()); + } + assert_violated(ViolatedProperty::MemsafetyDeref, 10, "main"); +} + +#[test] +fn load_after_free_is_invalid() { + // develop: the freed row has size 0, so no live row covers the address + // (AllocationLog.c:127-148, AnalysisModeMemtrack.c:156-162, :360-363). + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x2000), 16); + map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_check_load(addr(0x2004), 10, 0, 4, c"main".as_ptr()); + } + assert_violated(ViolatedProperty::MemsafetyDeref, 10, "main"); +} + +#[test] +fn reallocation_at_same_address_is_valid_again() { + // develop: `map2check_malloc` revives the existing row (is_free = FALSE, + // new size; AnalysisModeMemtrack.c:313-317). + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x2000), 16); + map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_map_malloc(addr(0x2000), 16); + map2check_check_load(addr(0x2004), 10, 0, 4, c"main".as_ptr()); + map2check_check_deref(addr(0x2000), 0, 11, c"main".as_ptr()); + map2check_check_free(c"p".as_ptr(), addr(0x2000), 0, 12, c"main".as_ptr()); + map2check_check_free_resolved_address(addr(0x2000), 13, c"main".as_ptr(), 0); + } + assert_ok(); + + // SAFETY: as above. + unsafe { + map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 14, c"main".as_ptr()); + map2check_check_load(addr(0x2004), 15, 0, 4, c"main".as_ptr()); + } + assert_violated(ViolatedProperty::MemsafetyDeref, 15, "main"); +} + +#[test] +fn reallocation_with_smaller_size_shrinks_the_valid_range() { + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x2000), 16); + map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_map_malloc(addr(0x2000), 8); + map2check_check_load(addr(0x2004), 10, 0, 4, c"main".as_ptr()); + } + assert_ok(); + // SAFETY: as above. + unsafe { map2check_check_load(addr(0x2008), 11, 0, 4, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::MemsafetyDeref, 11, "main"); +} + +#[test] +fn load_range_boundaries_of_a_live_allocation() { + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x2000), 16); + map2check_check_load(addr(0x2000), 1, 0, 16, c"main".as_ptr()); + map2check_check_load(addr(0x200c), 2, 0, 4, c"main".as_ptr()); + map2check_check_load(addr(0x200f), 3, 0, 1, c"main".as_ptr()); + } + assert_ok(); + + for (address, size, line) in [(0x1fff, 1, 4), (0x200d, 4, 5), (0x2010, 1, 6)] { + state::reset(); + // SAFETY: as above. + unsafe { + map2check_map_malloc(addr(0x2000), 16); + map2check_check_load(addr(address), line, 0, size, c"main".as_ptr()); + } + assert_violated(ViolatedProperty::MemsafetyDeref, line as u32, "main"); + } +} + +#[test] +fn pointer_store_does_not_revive_a_freed_block() { + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x2000), 16); + map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_map_store_pointer( + addr(0x3000), + addr(0x2000), + 0, + c"q".as_ptr(), + 10, + c"main".as_ptr(), + ); + map2check_check_load(addr(0x2000), 11, 0, 4, c"main".as_ptr()); + } + assert_violated(ViolatedProperty::MemsafetyDeref, 11, "main"); +} + #[test] fn check_free_accepts_live_allocation_and_null() { let _guard = fresh(); diff --git a/rust/src/memtrack/mod.rs b/rust/src/memtrack/mod.rs index 3228aba..8634c18 100644 --- a/rust/src/memtrack/mod.rs +++ b/rust/src/memtrack/mod.rs @@ -152,19 +152,40 @@ pub fn has_memcleanup_error(entries: &[MemTrackEntry]) -> bool { }) } -/// Checks whether `address` is a valid allocation (within any live allocation's range). +/// Checks whether the `size` bytes at `address` lie outside every valid block. /// Replaces `is_a_invalid_address_in_cntr`. +/// +/// Follows `map2check_load` of the Map2Check `develop` C runtime +/// (`AnalysisModeMemtrack.c:156-162`), which accepts the access when a stack +/// slot covers it (`is_valid_heap_address`, `HeapLog.c:37-52`) or a live heap +/// block does (`is_valid_allocation_address`, `AllocationLog.c:127-148`): +/// +/// - a stack slot (`map2check_map_alloca`, `_non_static_alloca`, +/// `_funct_address`: neither dynamic nor freed, non-zero size) stays valid; +/// - for heap blocks, only the latest alloc/free event per block address +/// counts (#11): `develop` keeps one row per address and updates it in place +/// on malloc and free (`AnalysisModeMemtrack.c:313-317`, `:360-363`), so a +/// free invalidates the block and a later malloc at the same address makes +/// it valid again, with the new size; +/// - pointer stores (`map2check_map_store_pointer`, size 0) are not events. pub fn is_invalid_address(entries: &[MemTrackEntry], address: usize, size: usize) -> bool { let Some(address_end) = address.checked_add(size) else { return true; }; - !entries.iter().rev().any(|entry| { - !entry.is_free - && address >= entry.points_to + let covers = |entry: &MemTrackEntry| { + address >= entry.points_to && entry .points_to .checked_add(entry.size_destiny) .is_some_and(|allocation_end| address_end <= allocation_end) + }; + let mut decided = HashSet::new(); + !entries.iter().rev().any(|entry| { + if entry.is_dynamic || entry.is_free { + decided.insert(entry.points_to) && !entry.is_free && covers(entry) + } else { + entry.size_destiny > 0 && covers(entry) + } }) } @@ -324,6 +345,29 @@ mod tests { assert!(!is_deref_error(&entries, 0x200)); } + #[test] + fn invalid_address_uses_latest_event_per_block() { + let malloc = make_entry(0x200, 0x200, true, false, 16); + let mut free = make_entry(0x200, 0x200, true, false, 16); + free.set_free(); + assert!(is_invalid_address( + &[malloc.clone(), free.clone()], + 0x204, + 4 + )); + let realloc = make_entry(0x200, 0x200, true, false, 8); + let live = [malloc, free, realloc]; + assert!(!is_invalid_address(&live, 0x204, 4)); + assert!(is_invalid_address(&live, 0x208, 4)); + } + + #[test] + fn invalid_address_keeps_stack_slots_valid() { + let entries = vec![make_entry(0x300, 0x300, false, false, 8)]; + assert!(!is_invalid_address(&entries, 0x304, 4)); + assert!(is_invalid_address(&entries, 0x306, 4)); + } + #[test] fn invalid_address_rejects_wrapping_ranges() { let entries = vec![make_entry(0x100, usize::MAX - 3, true, false, 16)]; From 91f82f4ce87191cf9056f56f2645bd99edfe536a Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 21:10:29 -0400 Subject: [PATCH 16/63] fix: record the release when a resolved free passes (#11, #37) map2check_check_free_resolved_address only checked, while map2check_free_resolved_address in the C runtime of Map2Check develop also marks the block freed (AnalysisModeMemtrack.c:178-182). The pass uses this path for realloc and for frees of a non-load value, so a moved realloc gave a false MEMSAFETY_MEMCLEANUP, and use-after-free and double free through this path went unreported. The export now records a free event when the check passes (never for NULL or a failed check). Add tests for a realloc-style sequence, a double resolved free and a load after a resolved free. Update ABI mapping row 27 and document the asymmetry with map2check_check_free. Also add SD-13 (intentional): a block freed without ever being stored through a tracked pointer is a valid free here, while develop reports FALSE_FREE because is_invalid_free scans only list_log. Correct SD-06, which overstated parity, and document vcc_reset_meta_data as a host/test-only export that clears a violation by design. --- docs/api-deltas.md | 30 +++++++++++------- docs/cutover/abi-mapping.tsv | 2 +- docs/semantic-deltas.md | 42 +++++++++++++++++++++++-- rust/src/ffi.rs | 39 ++++++++++++++++++++--- rust/src/ffi_tests.rs | 60 ++++++++++++++++++++++++++++++++++++ 5 files changed, 155 insertions(+), 18 deletions(-) diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 21603a0..87048c9 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -51,7 +51,7 @@ build with `--features libfuzzer`. | `get_current_step` | `uint64_t get_current_step(void)` | `long get_current_step(void)` | changed | Returns `uint64_t` instead of `long`. | | `get_next_step` | `uint64_t get_next_step(void)` | `get_next_step()` / `next_current_step()` | same | Advances and returns the step counter. `next_current_step` is not exported; see its row. | | `print_all_containers_as_json` | `void print_all_containers_as_json(void)` | `print_all_containers_as_json` | changed | Report shape and whitespace differ (SD-01, SD-02; schema decision in #14). | -| `vcc_reset_meta_data` | `void vcc_reset_meta_data(void)` | `vcc_reset_meta_data` | same | | +| `vcc_reset_meta_data` | `void vcc_reset_meta_data(void)` | `vcc_reset_meta_data` | same | Host/test-only: the Map2Check passes never call it. It clears a recorded violation by design, bypassing the first-violation rule (SD-03). | | `map2check_set_memcleanup` | `void map2check_set_memcleanup(void)` | `map2check_set_memcleanup` | same | Enables the end-of-program leak check (SD-04). | | `map2check_set_null_is_valid` | `void map2check_set_null_is_valid(void)` | `map2check_set_null_is_valid` | same | Makes a NULL load safe (SD-05). | | `map2check_save_nondet_log_int` | `non_det_log_t *map2check_save_nondet_log_int(int line, int scope, enum var_type_t type_var, int *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | @@ -84,7 +84,7 @@ build with `--features libfuzzer`. | `map2check_binop_shr_unsigned` | `void map2check_binop_shr_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shr_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | | `map2check_check_load` | `void map2check_check_load(const void *ptr, int32_t line, uint32_t scope, int32_t size, const char *function_name)` | `map2check_check_load` | changed | Address-validity rule rewritten (plan R1; differential check in #15); since #11 a heap block is valid only while its latest alloc/free event is an allocation, as in `develop` (`AllocationLog.c:127-148`). The first violation is final; a later check never clears or replaces it (SD-03); NULL handling in SD-05. | | `map2check_check_free` | `void map2check_check_free(const char *_name, const void *ptr, uint32_t _scope, uint32_t line, const char *function_name)` | `map2check_check_free` | same | `_name` and `_scope` are ignored. Untracked address is a violation (SD-06). | -| `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; untracked address is a violation (SD-06). | +| `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; records the release when the check passes, unlike `map2check_check_free`; untracked address is a violation (SD-06); see also SD-13. | | `map2check_check_deref` | `void map2check_check_deref(const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_deref` | same | Name collides with the pass ABI `map2check_check_deref(i64, ptr)`; see the pass matrix (#21). | | `map2check_check_mem_endprog` | `void map2check_check_mem_endprog(void)` | `map2check_check_mem_endprog` | changed | Leak algorithm rewritten; no-op until `map2check_set_memcleanup()` (SD-04). | | `LLVMFuzzerTestOneInput` | `int32_t LLVMFuzzerTestOneInput(const uint8_t *data, uintptr_t size)` | `LLVMFuzzerTestOneInput` | same | Feature `libfuzzer`. Resets the analysis state on every iteration (SD-11). | @@ -263,12 +263,20 @@ receives. The Rust export: - reports `MEMSAFETY_FREE` when the most recent record whose `mem_address_points_to` equals `ptr` is already freed; - reports `MEMSAFETY_FREE` for an address with no heap record (SD-06, #37); -- only checks: it does not mark the block freed. The caller records the - release with `map2check_map_free`. - -The C runtime of Map2Check `develop` -(`modules/backend/library/lib/AnalysisModeMemtrack.c:164-183` and -`ListLog.c:95-118`) agrees on the untracked address since #37, and still -differs in one way: the function also marks the block freed. That keeps the -split between checking and recording that the rest of the core API already -uses. +- records the release (a free event for `ptr`) when the check passes, so + the caller must not add `map2check_map_free`. A NULL `ptr` and a failed + check record nothing. + +This matches the C runtime of Map2Check `develop` +(`modules/backend/library/lib/AnalysisModeMemtrack.c:164-183`): the +function checks with `is_invalid_free` (`ListLog.c:95-118`) and then marks +the block freed (`:178-182`). The pass uses it for `realloc` and for frees +whose argument is not a load, so it is a single call. + +**Asymmetry with `map2check_check_free`:** `map2check_check_free` only +checks; the pass calls `map2check_map_free` after it to record the +release. `map2check_check_free_resolved_address` checks and records in one +call. + +The free check itself consults the allocation and free records, not the +pointer stores that `develop` scans; see SD-13. diff --git a/docs/cutover/abi-mapping.tsv b/docs/cutover/abi-mapping.tsv index 4b0617e..d791009 100644 --- a/docs/cutover/abi-mapping.tsv +++ b/docs/cutover/abi-mapping.tsv @@ -24,7 +24,7 @@ map2check_add_store_pointer map2check_map_store_pointer - C: (ptr var,ptr value, map2check_load map2check_check_load - C: (ptr,i64 size) (MemoryTrackPass.cpp:770) → núcleo: (ptr,line:i32,scope:u32,size:i32,fn); o passe passa a emitir linha, escopo e função (DebugInfo); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (a flag ERROR_DEREF de AnalysisModeMemtrack.c:158 só encerra em check_deref); check_load grava a violação direto map2check_check_deref - - check_load já registra a violação (ffi.rs:894, LoadChecker); C: (i64 line,ptr fn) (MemoryTrackPass.cpp:785) só lê a flag ERROR_DEREF e encerra (AnalysisModeMemtrack.c:297); o passe deixa de emitir; C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) map2check_free map2check_check_free+map2check_map_free - C: (ptr name,ptr var_slot,i64 scope,i64 line,ptr fn) (MemoryTrackPass.cpp:818) → núcleo (ambos): (name,ptr,scope:u32,line:u32,fn); semântica confirmada: o núcleo é chaveado no endereço de heap carregado (ffi_tests.rs:722-744), então o passe carrega *ptr antes da chamada e a ordem é check_free depois map_free; o passe não chama nenhum dos dois se o valor carregado for NULL, como o C (AnalysisModeMemtrack.c:342); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:352) -map2check_free_resolved_address map2check_check_free_resolved_address - SD-06 (#37, corrigido): endereço sem registro reporta MEMSAFETY_FREE, como is_invalid_free (ListLog.c:95-118); C: (ptr,i64 line,ptr fn,i1 nullValid) (MemoryTrackPass.cpp:774) → núcleo: (ptr,line:u32,fn,is_null_valid:i16); i1 passa a i16 (zext); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:170) +map2check_free_resolved_address map2check_check_free_resolved_address - chamada única; registra a liberação quando a checagem passa (como o C, :178-182); check_free não registra — o passe chama map_free depois; vale também para a instrumentação de realloc (MemoryTrackPass.cpp:169-182); SD-06 (#37, corrigido): endereço sem registro reporta MEMSAFETY_FREE, como is_invalid_free (ListLog.c:95-118); SD-13: free de ponteiro nunca armazenado é válido no núcleo e FALSE_FREE no C; C: (ptr,i64 line,ptr fn,i1 nullValid) (MemoryTrackPass.cpp:774) → núcleo: (ptr,line:u32,fn,is_null_valid:i16); i1 passa a i16 (zext); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:170) map2check_posix map2check_map_posix_memalign 2.5 chamado em MemoryTrackPass.cpp:120 para posix_memalign; não é obsoleto; C: (ptr,i64 size) (MemoryTrackPass.cpp:793); o C runtime desreferencia ptr (void**) e registra malloc+store_pointer (AnalysisModeMemtrack.c:328); núcleo proposto: (ptr_slot,size:i32) map2check_wasm_malloc map2check_wasm_malloc 2.8 C: (i64,i64) (MemoryTrackPass.cpp:825) map2check_wasm_free map2check_wasm_free 2.8 C: (i64,i64) (MemoryTrackPass.cpp:829) diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index ce3ff26..dabd156 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -175,7 +175,8 @@ map2check_check_load(NULL, 12, 0, 4, "main"); /* None */ both report `MEMSAFETY_FREE` for an address with no heap record (no allocation or free event; a stack slot or a pointer store does not count). The most recent heap record for the address decides: a freed block is an -invalid free, a live one is valid. +invalid free, a live one is valid. When the resolved check passes, it also +records the release, as `develop` does (`AnalysisModeMemtrack.c:178-182`). Before #37, `map2check_check_free_resolved_address` treated an address with no record as a safe free (`is_invalid_free_resolved` ended in @@ -187,7 +188,14 @@ lets the most recent `list_log` row pointing to the address decide (invalid if freed or not dynamic) and returns true when no row is found. Both `map2check_free` (`AnalysisModeMemtrack.c:351`) and `map2check_free_resolved_address` (`AnalysisModeMemtrack.c:171`) use it in -memtrack mode. `develop` matches `lib_old` here. (In memcleanup mode, +memtrack mode. `develop` matches `lib_old` here. + +Parity is on the verdict for an untracked address only, not on the lookup. +`develop` scans `list_log` (pointer stores, free rows and reference +updates) and never `allocation_log`. Rust scans the allocation and free +records and skips pointer stores, which carry no status snapshot here. The +two differ when a block is freed without ever being stored through a +tracked pointer store; that case is SD-13. (In memcleanup mode, `AnalysisModeMemcleanup.c:167` and `:341` hard-code `error = FALSE`; this library does not have that mode split.) @@ -334,3 +342,33 @@ map2check_set_memcleanup(); map2check_check_mem_endprog(); /* None (before #38: MemsafetyMemcleanup) */ map2check_check_load(&vla[2], 4, 0, sizeof(int), "main"); /* None (before #38: MemsafetyDeref) */ ``` + +### SD-13 — Freeing a block that was never stored through a pointer + +**Classification:** intentional +**Tests:** `rust/src/ffi_tests.rs::sd13_free_of_a_never_stored_pointer_is_valid` + +**Rust:** the free check decides by the allocation and free records. A live +block is a valid free even if its address was never written to a tracked +pointer variable, so `free(malloc(8))` reports nothing. This is the correct +verdict: the program frees a live heap block exactly once. + +**C runtime of Map2Check `develop`:** `is_invalid_free` +(`modules/backend/library/lib/ListLog.c:95-118`) only scans `list_log`, +which `map2check_add_store_pointer` fills (`AnalysisModeMemtrack.c:246-295`) +together with free rows and reference updates (`:185-244`). `map2check_malloc` +writes only `allocation_log` (`:305-323`). When the freed pointer was never +stored through a tracked store, there is no row, `is_invalid_free` returns +TRUE and `map2check_free_resolved_address` reports FALSE_FREE +(`AnalysisModeMemtrack.c:171-177`). This happens for `free(malloc(8))`, or +for `free(mk())` where `mk` returns `malloc(n)`. At -O0 the result is not +stored, and the pass takes the resolved path (`MemoryTrackPass.cpp:311-327`). +This is a false positive in `develop`. + +**Reproduce:** + +```c +map2check_init(); +map2check_map_malloc((void *)0x1000, 8); +map2check_check_free_resolved_address((void *)0x1000, 3, "main", 0); /* None; develop: FALSE_FREE */ +``` diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index ad94837..cccf0a6 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -56,7 +56,9 @@ fn log_error(e: &Map2CheckError) { eprintln!("[map2check] error: {e}"); } -fn run_vcc_check(checker: impl VccChecker, ctx: &VccContext, line: u32, fname: &str) { +/// Runs `checker`, records a violation if it finds one, and returns whether +/// the check passed (`false` on a violation or an internal error). +fn run_vcc_check(checker: impl VccChecker, ctx: &VccContext, line: u32, fname: &str) -> bool { match checker.check(ctx) { Ok(crate::analysismode::VccOutcome::Violated { property }) => { if let Err(e) = state::with_state(|s| { @@ -65,9 +67,13 @@ fn run_vcc_check(checker: impl VccChecker, ctx: &VccContext, line: u32, fname: & }) { log_error(&e); } + false + } + Ok(crate::analysismode::VccOutcome::Safe) => true, + Err(e) => { + log_error(&e); + false } - Ok(crate::analysismode::VccOutcome::Safe) => {} - Err(e) => log_error(&e), } } @@ -207,6 +213,8 @@ pub extern "C" fn print_all_containers_as_json() { } /// Resets violation metadata without clearing tracking containers. +/// Host/test-only: the Map2Check passes never call it. It clears a recorded +/// violation by design, bypassing the first-violation rule (#36, SD-03). #[no_mangle] pub extern "C" fn vcc_reset_meta_data() { ffi_guard( @@ -968,6 +976,14 @@ pub unsafe extern "C" fn map2check_check_free( /// the heap address itself (post-indirection), not the address of the /// pointer variable that holds it (that case is `map2check_check_free`). /// +/// When the check passes, it also records the release (a free event for +/// `ptr`), as `map2check_free_resolved_address` of the Map2Check `develop` C +/// runtime does (`AnalysisModeMemtrack.c:178-182`). The pass uses this path +/// for `realloc` and for frees of a value that is not a load, so the caller +/// must not add `map2check_map_free`. Unlike this export, +/// `map2check_check_free` only checks; the pass calls `map2check_map_free` +/// after it. A NULL `ptr` is never recorded. +/// /// # Safety /// `function_name` must be null or a valid null-terminated C string; `ptr` may be null. #[no_mangle] @@ -985,7 +1001,7 @@ pub unsafe extern "C" fn map2check_check_free_resolved_address( return; } let ctx = VccContext::new(line, 0, fname); - run_vcc_check( + let passed = run_vcc_check( FreeResolvedChecker { address: ptr as usize, }, @@ -993,6 +1009,21 @@ pub unsafe extern "C" fn map2check_check_free_resolved_address( line, fname, ); + if !passed || ptr.is_null() { + return; + } + let addr = ptr as usize; + if let Err(e) = state::with_state(|s| { + let step = s.next_step(); + let mut entry = MemTrackEntry::new( + step, line, 0, addr, addr, true, false, "", fname, 0, 1, false, + ); + entry.set_free(); + s.memtrack.push(entry); + Ok(()) + }) { + log_error(&e); + } }, (), ); diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 13c69c1..694f792 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -986,6 +986,66 @@ fn check_free_resolved_address_rejects_stack_slot() { assert_violated(ViolatedProperty::MemsafetyFree, 5, "main"); } +#[test] +fn resolved_free_records_the_release_so_a_moved_realloc_is_not_a_leak() { + // develop: `map2check_free_resolved_address` marks the block freed when + // the check passes (AnalysisModeMemtrack.c:178-182). The pass uses it for + // realloc: free the old block, then malloc the new one. + let _guard = fresh(); + // SAFETY: C-string literal is valid; fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x1000), 16); + map2check_check_free_resolved_address(addr(0x1000), 3, c"main".as_ptr(), 0); + map2check_map_malloc(addr(0x2000), 32); + map2check_check_free_resolved_address(addr(0x2000), 4, c"main".as_ptr(), 0); + } + map2check_set_memcleanup(); + map2check_check_mem_endprog(); + assert_ok(); +} + +#[test] +fn resolved_free_twice_is_an_invalid_free() { + let _guard = fresh(); + // SAFETY: C-string literal is valid; fake address is never dereferenced. + unsafe { + map2check_map_malloc(addr(0x1000), 16); + map2check_check_free_resolved_address(addr(0x1000), 3, c"main".as_ptr(), 0); + } + assert_ok(); + // SAFETY: as above. + unsafe { map2check_check_free_resolved_address(addr(0x1000), 4, c"main".as_ptr(), 0) }; + assert_violated(ViolatedProperty::MemsafetyFree, 4, "main"); +} + +#[test] +fn load_after_resolved_free_is_invalid() { + let _guard = fresh(); + // SAFETY: C-string literal is valid; fake address is never dereferenced. + unsafe { + map2check_map_malloc(addr(0x1000), 16); + map2check_check_free_resolved_address(addr(0x1000), 3, c"main".as_ptr(), 0); + map2check_check_load(addr(0x1000), 4, 0, 4, c"main".as_ptr()); + } + assert_violated(ViolatedProperty::MemsafetyDeref, 4, "main"); +} + +#[test] +fn sd13_free_of_a_never_stored_pointer_is_valid() { + // SD-13: `free(malloc(8))` — the block was never stored through a tracked + // pointer store. Rust decides by the allocation record and accepts it; + // develop scans only list_log (pointer stores) and reports FALSE_FREE. + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x1000), 8); + map2check_check_free_resolved_address(addr(0x1000), 3, c"main".as_ptr(), 0); + map2check_map_malloc(addr(0x2000), 8); + map2check_check_free(c"p".as_ptr(), addr(0x2000), 0, 4, c"main".as_ptr()); + } + assert_ok(); +} + #[test] fn check_free_resolved_address_null_depends_on_flag() { let _guard = fresh(); From f782db4a1d9de076fe2f29bf9e840046c1a7e762 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 21:14:36 -0400 Subject: [PATCH 17/63] fix: honour memcleanup and null-is-valid options in memory checks (#10) Keep the options in a configuration that init and reset do not clear. The end-of-program leak check now always runs and reports memtrack or memcleanup by option, as the develop C runtime does (SD-04). --- docs/api-deltas.md | 10 +-- docs/semantic-deltas.md | 57 +++++++++------ rust/src/analysismode/memory.rs | 22 ++++-- rust/src/ffi.rs | 45 ++++++------ rust/src/ffi_tests.rs | 121 +++++++++++++++++++++++++++++--- rust/src/state.rs | 53 ++++++++++++-- 6 files changed, 240 insertions(+), 68 deletions(-) diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 87048c9..32c05eb 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -43,8 +43,8 @@ build with `--features libfuzzer`. | Symbol | Rust declaration (`include/map2check.h`) | Legacy C library | Status | Notes | |---|---|---|---|---| -| `map2check_init` | `void map2check_init(void)` | `map2check_init` | same | Map2Check passes call `map2check_init(int)`; see the pass matrix. | -| `map2check_reset` | `void map2check_reset(void)` | — | new | Clears containers, result, step counter and flags (#9). | +| `map2check_init` | `void map2check_init(void)` | `map2check_init` | same | Starts a fresh analysis (same as reset) and keeps the options (#10). Map2Check passes call `map2check_init(int)`; see the pass matrix. | +| `map2check_reset` | `void map2check_reset(void)` | — | new | Clears containers, result and step counter (#9). Keeps the options set by `map2check_set_memcleanup()` and `map2check_set_null_is_valid()`: they are configuration, not analysis state (#10). | | `map2check_success` | `void map2check_success(void)` | `map2check_success` | same | Prints the JSON report (SD-01). | | `set_false_result` | `void set_false_result(int32_t prp, int32_t line_number, const char *function_name)` | `set_false_result` | same | `ViolatedProperty` keeps the C variant order (`caller_lib_result.h:22-31`); unknown values are ignored (SD-09). Ignored when a violation is already recorded (SD-03). | | `get_current_property` | `enum ViolatedProperty get_current_property(void)` | `get_current_property` | same | | @@ -52,8 +52,8 @@ build with `--features libfuzzer`. | `get_next_step` | `uint64_t get_next_step(void)` | `get_next_step()` / `next_current_step()` | same | Advances and returns the step counter. `next_current_step` is not exported; see its row. | | `print_all_containers_as_json` | `void print_all_containers_as_json(void)` | `print_all_containers_as_json` | changed | Report shape and whitespace differ (SD-01, SD-02; schema decision in #14). | | `vcc_reset_meta_data` | `void vcc_reset_meta_data(void)` | `vcc_reset_meta_data` | same | Host/test-only: the Map2Check passes never call it. It clears a recorded violation by design, bypassing the first-violation rule (SD-03). | -| `map2check_set_memcleanup` | `void map2check_set_memcleanup(void)` | `map2check_set_memcleanup` | same | Enables the end-of-program leak check (SD-04). | -| `map2check_set_null_is_valid` | `void map2check_set_null_is_valid(void)` | `map2check_set_null_is_valid` | same | Makes a NULL load safe (SD-05). | +| `map2check_set_memcleanup` | `void map2check_set_memcleanup(void)` | `map2check_set_memcleanup` | same | Selects `MEMSAFETY_MEMCLEANUP` instead of `MEMSAFETY_MEMTRACK` for the end-of-program leak (SD-04). Kept by `map2check_init()` and `map2check_reset()`. | +| `map2check_set_null_is_valid` | `void map2check_set_null_is_valid(void)` | `map2check_set_null_is_valid` | same | Makes a NULL load safe (SD-05). Kept by `map2check_init()` and `map2check_reset()`. | | `map2check_save_nondet_log_int` | `non_det_log_t *map2check_save_nondet_log_int(int line, int scope, enum var_type_t type_var, int *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_uint` | `non_det_log_t *map2check_save_nondet_log_uint(int line, int scope, enum var_type_t type_var, unsigned int *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_long` | `non_det_log_t *map2check_save_nondet_log_long(int line, int scope, enum var_type_t type_var, long *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | @@ -86,7 +86,7 @@ build with `--features libfuzzer`. | `map2check_check_free` | `void map2check_check_free(const char *_name, const void *ptr, uint32_t _scope, uint32_t line, const char *function_name)` | `map2check_check_free` | same | `_name` and `_scope` are ignored. Untracked address is a violation (SD-06). | | `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; records the release when the check passes, unlike `map2check_check_free`; untracked address is a violation (SD-06); see also SD-13. | | `map2check_check_deref` | `void map2check_check_deref(const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_deref` | same | Name collides with the pass ABI `map2check_check_deref(i64, ptr)`; see the pass matrix (#21). | -| `map2check_check_mem_endprog` | `void map2check_check_mem_endprog(void)` | `map2check_check_mem_endprog` | changed | Leak algorithm rewritten; no-op until `map2check_set_memcleanup()` (SD-04). | +| `map2check_check_mem_endprog` | `void map2check_check_mem_endprog(void)` | `map2check_check_mem_endprog` | changed | Leak algorithm rewritten; always runs, reporting memtrack or memcleanup by option (SD-04). | | `LLVMFuzzerTestOneInput` | `int32_t LLVMFuzzerTestOneInput(const uint8_t *data, uintptr_t size)` | `LLVMFuzzerTestOneInput` | same | Feature `libfuzzer`. Resets the analysis state on every iteration (SD-11). | | `__VERIFIER_nondet_bool` | `bool __VERIFIER_nondet_bool(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | | `__VERIFIER_nondet_char` | `int8_t __VERIFIER_nondet_char(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index dabd156..778ff47 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -121,41 +121,56 @@ map2check_check_load((void *)0x1000, 2, 0, 4, "main"); /* get_current_property() == Reachability (before #36: None) */ ``` -### SD-04 — End-of-program leak check is opt-in - -**Classification:** intentional -**Tests:** `rust/src/ffi_tests.rs::check_mem_endprog_is_a_no_op_until_memcleanup_is_enabled`, `rust/src/ffi_tests.rs::set_memcleanup_enables_end_of_program_check` - -**Rust:** `map2check_check_mem_endprog()` does nothing until -`map2check_set_memcleanup()` has been called. After that call, a dynamic -allocation that was never freed reports `MEMSAFETY_MEMCLEANUP`. - -**Legacy (embedded C library):** `map2check_exit()` calls -`analysis_is_program_correct()` at the end of every run -(`lib_old/Map2CheckFunctions.c:89-102`). Whether a flag gated it was not -verified (#15). +### SD-04 — End-of-program leak check runs in both modes + +**Classification:** fixed (#10) +**Tests:** `rust/src/ffi_tests.rs::without_memcleanup_leak_is_reported_as_memtrack_at_exit`, `rust/src/ffi_tests.rs::memcleanup_reports_leak_as_memcleanup_at_exit`, `rust/src/ffi_tests.rs::endprog_keeps_an_earlier_violation`, `rust/src/ffi_tests.rs::options_survive_init_and_reset` + +**Rust:** `map2check_check_mem_endprog()` always runs the leak check. A +dynamic allocation that was never freed reports `MEMSAFETY_MEMCLEANUP` after +`map2check_set_memcleanup()` and `MEMSAFETY_MEMTRACK` otherwise. The option +is configuration: `map2check_init()` and `map2check_reset()` keep it. Until +the analysis mode is configurable (Task 2.3) the memcleanup option picks +between the memtrack and memcleanup verdicts. A violation recorded earlier +stays final (SD-03). + +**Before #10:** the Rust function did nothing until +`map2check_set_memcleanup()` had been called, so a leak in memtrack mode went +unreported. + +**C runtime of Map2Check `develop`:** `map2check_exit()` calls +`analysis_is_program_correct()` when no error was recorded +(`Map2CheckFunctions.c:89-92`). In memtrack mode it writes `FALSE_MEMTRACK` +(`AnalysisModeMemtrack.c:138-146`, tag at `:141`); in memcleanup mode it +writes `FALSE_MEMCLEANUP` (`AnalysisModeMemcleanup.c:138-146`, tag at `:142`). +Both use the same `valid_allocation_log()` scan, and both report line 0 with +an empty function name. **Reproduce:** ```c map2check_init(); map2check_map_malloc((void *)0x1000, 16); -map2check_check_mem_endprog(); /* None */ -map2check_set_memcleanup(); -map2check_check_mem_endprog(); /* MemsafetyMemcleanup */ +map2check_check_mem_endprog(); /* MemsafetyMemtrack */ +/* with map2check_set_memcleanup() first: MemsafetyMemcleanup */ ``` ### SD-05 — NULL load is a violation unless NULL is declared valid **Classification:** intentional -**Tests:** `rust/src/ffi_tests.rs::check_load_null_depends_on_null_is_valid` +**Tests:** `rust/src/ffi_tests.rs::check_load_null_depends_on_null_is_valid`, `rust/src/ffi_tests.rs::null_is_valid_accepts_null_load`, `rust/src/ffi_tests.rs::options_survive_init_and_reset` **Rust:** `map2check_check_load(NULL, ...)` reports `MEMSAFETY_DEREF` unless `map2check_set_null_is_valid()` was called first. After that call it is -safe. - -**Legacy (embedded C library):** `map2check_load(ptr, size)` has no NULL -case and no flag (`lib_old/AnalysisModeMemtrack.c:156-162`). +safe. The option survives `map2check_init()` and `map2check_reset()`. + +**C runtime of Map2Check `develop`:** there is no NULL-is-valid notion on +load. `map2check_load(ptr, size)` finds NULL in neither the heap log nor the +allocation log and sets `ERROR_DEREF` (`AnalysisModeMemtrack.c:156-162`); +`map2check_check_deref` then reports `FALSE_DEREF` (`:297-301`). The default +(option off) matches that verdict. The only NULL flag in `develop` is +`isNullValid` of `map2check_free_resolved_address`, for frees +(`:164-169`). The load option is a Rust extension, hence intentional. **Reproduce:** diff --git a/rust/src/analysismode/memory.rs b/rust/src/analysismode/memory.rs index 4b85aaa..41c9d4e 100644 --- a/rust/src/analysismode/memory.rs +++ b/rust/src/analysismode/memory.rs @@ -126,14 +126,21 @@ impl VccChecker for DerefChecker { /// Checks for memory leaks at end of program. /// Replaces `map2check_check_mem_endprog`. -pub struct MemCleanupChecker; +pub struct MemCleanupChecker { + /// `true` reports `MemsafetyMemcleanup`, `false` `MemsafetyMemtrack`. + pub memcleanup: bool, +} impl VccChecker for MemCleanupChecker { fn check(&self, _ctx: &VccContext) -> Result { state::with_state(|s| { if memtrack::has_memcleanup_error(&s.memtrack) { Ok(VccOutcome::Violated { - property: ViolatedProperty::MemsafetyMemcleanup, + property: if self.memcleanup { + ViolatedProperty::MemsafetyMemcleanup + } else { + ViolatedProperty::MemsafetyMemtrack + }, }) } else { Ok(VccOutcome::Safe) @@ -337,7 +344,9 @@ mod tests { fn mem_cleanup_flags_leaked_allocation() { let _g = with_entries(vec![stack(0x30), heap(0x10, 0x1000, 16, false)]); assert_eq!( - MemCleanupChecker.check(&ctx()).unwrap(), + MemCleanupChecker { memcleanup: true } + .check(&ctx()) + .unwrap(), violated(ViolatedProperty::MemsafetyMemcleanup) ); } @@ -349,6 +358,11 @@ mod tests { heap(0x10, 0x1000, 16, false), heap(0x10, 0x1000, 16, true), ]); - assert_eq!(MemCleanupChecker.check(&ctx()).unwrap(), VccOutcome::Safe); + assert_eq!( + MemCleanupChecker { memcleanup: true } + .check(&ctx()) + .unwrap(), + VccOutcome::Safe + ); } } diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index cccf0a6..ee48db3 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -233,37 +233,29 @@ pub extern "C" fn vcc_reset_meta_data() { ); } -/// Enables memcleanup checking at end of program. -/// Effect: `map2check_check_mem_endprog` only runs the leak check once this -/// has been called; before that it is a no-op. +/// Selects the memcleanup property for the end-of-program leak check. +/// Effect: `map2check_check_mem_endprog` reports a leak as +/// `MemsafetyMemcleanup` instead of `MemsafetyMemtrack`. The option is +/// configuration: `map2check_init()` and `map2check_reset()` keep it. #[no_mangle] pub extern "C" fn map2check_set_memcleanup() { ffi_guard( || { - if let Err(e) = state::with_state(|s| { - s.memcleanup_enabled = true; - Ok(()) - }) { - log_error(&e); - } + state::update_config(|c| c.memcleanup = true); }, (), ); } -/// Marks NULL as valid for the current scope's pointer checks. +/// Marks NULL as valid for load checks. /// Effect: `map2check_check_load` treats a NULL `ptr` as safe once this has -/// been called, instead of always flagging it as `MemsafetyDeref`. +/// been called, instead of flagging it as `MemsafetyDeref`. Kept across +/// `map2check_init()` and `map2check_reset()`. #[no_mangle] pub extern "C" fn map2check_set_null_is_valid() { ffi_guard( || { - if let Err(e) = state::with_state(|s| { - s.null_is_valid = true; - Ok(()) - }) { - log_error(&e); - } + state::update_config(|c| c.null_is_valid = true); }, (), ); @@ -926,7 +918,7 @@ pub unsafe extern "C" fn map2check_check_load( return; } let ctx = VccContext::new(line as u32, scope, fname); - let is_null_valid = state::with_state(|s| Ok(s.null_is_valid)).unwrap_or(false); + let is_null_valid = state::config().null_is_valid; run_vcc_check( LoadChecker { address: ptr as usize, @@ -1058,17 +1050,22 @@ pub unsafe extern "C" fn map2check_check_deref( ); } -/// Checks for memory leaks at end of program. -/// No-op unless `map2check_set_memcleanup()` was called first. +/// Checks for memory leaks at end of program, in both modes (SD-04). A leak is +/// `MemsafetyMemcleanup` after `map2check_set_memcleanup()`, otherwise +/// `MemsafetyMemtrack`, as in the `develop` C runtime. #[no_mangle] pub extern "C" fn map2check_check_mem_endprog() { ffi_guard( || { - if !state::with_state(|s| Ok(s.memcleanup_enabled)).unwrap_or(false) { - return; - } let ctx = VccContext::new(0, 0, "end_of_program"); - run_vcc_check(MemCleanupChecker, &ctx, 0, "end_of_program"); + run_vcc_check( + MemCleanupChecker { + memcleanup: state::config().memcleanup, + }, + &ctx, + 0, + "end_of_program", + ); }, (), ); diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 694f792..86181c6 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -14,6 +14,7 @@ use std::sync::MutexGuard; fn fresh() -> MutexGuard<'static, ()> { let guard = state::test_lock(); state::reset(); + state::reset_config(); guard } @@ -80,8 +81,6 @@ fn second_init_starts_from_clean_state() { assert!(s.nondets.is_empty()); assert!(s.bbtrack.is_empty()); assert!(s.legacy_nondet_logs.is_empty()); - assert!(!s.memcleanup_enabled); - assert!(!s.null_is_valid); assert_eq!(s.result.line, 0); assert!(s.result.function_name.is_empty()); }); @@ -103,8 +102,6 @@ fn reset_clears_records_result_step_and_flags() { snapshot(|s| { assert_eq!(s.current_step(), 0); assert!(s.memtrack.is_empty()); - assert!(!s.memcleanup_enabled); - assert!(!s.null_is_valid); }); } @@ -203,17 +200,17 @@ fn vcc_reset_meta_data_clears_result_but_keeps_records() { #[test] fn set_memcleanup_enables_end_of_program_check() { let _guard = fresh(); - assert!(!snapshot(|s| s.memcleanup_enabled)); + assert!(!state::config().memcleanup); map2check_set_memcleanup(); - assert!(snapshot(|s| s.memcleanup_enabled)); + assert!(state::config().memcleanup); } #[test] fn set_null_is_valid_sets_the_flag() { let _guard = fresh(); - assert!(!snapshot(|s| s.null_is_valid)); + assert!(!state::config().null_is_valid); map2check_set_null_is_valid(); - assert!(snapshot(|s| s.null_is_valid)); + assert!(state::config().null_is_valid); } // ── NonDet tracking ───────────────────────────────────────────────────────── @@ -1084,13 +1081,16 @@ fn check_deref_accepts_live_and_rejects_freed_null_or_untracked() { } #[test] -fn check_mem_endprog_is_a_no_op_until_memcleanup_is_enabled() { +fn check_mem_endprog_reports_memtrack_until_memcleanup_is_enabled() { let _guard = fresh(); // SAFETY: fake address is never dereferenced. unsafe { map2check_map_malloc(addr(0x1000), 16) }; map2check_check_mem_endprog(); - assert_ok(); + assert_violated(ViolatedProperty::MemsafetyMemtrack, 0, "end_of_program"); + state::reset(); + // SAFETY: fake address is never dereferenced. + unsafe { map2check_map_malloc(addr(0x1000), 16) }; map2check_set_memcleanup(); map2check_check_mem_endprog(); assert_violated(ViolatedProperty::MemsafetyMemcleanup, 0, "end_of_program"); @@ -1142,3 +1142,104 @@ fn macro_generated_exports_match_the_hand_written_header_declarations() { let _: Shift = map2check_binop_shr_int; let _: Shift = map2check_binop_shr_unsigned; } + +// ── Memory-safety options (#10, SD-04, SD-05) ─────────────────────────────── + +#[test] +fn memcleanup_reports_leak_as_memcleanup_at_exit() { + let _guard = fresh(); + map2check_init(); + map2check_set_memcleanup(); + // SAFETY: fake address is never dereferenced. + unsafe { map2check_map_malloc(addr(0x3000), 32) }; + map2check_check_mem_endprog(); + assert_violated(ViolatedProperty::MemsafetyMemcleanup, 0, "end_of_program"); +} + +#[test] +fn without_memcleanup_leak_is_reported_as_memtrack_at_exit() { + // develop: analysis_is_program_correct() of AnalysisModeMemtrack.c writes + // FALSE_MEMTRACK; the leak check runs at exit in both modes. + let _guard = fresh(); + map2check_init(); + // SAFETY: fake address is never dereferenced. + unsafe { map2check_map_malloc(addr(0x3000), 32) }; + map2check_check_mem_endprog(); + assert_violated(ViolatedProperty::MemsafetyMemtrack, 0, "end_of_program"); +} + +#[test] +fn endprog_without_leak_is_ok_in_both_modes() { + let _guard = fresh(); + map2check_check_mem_endprog(); + assert_ok(); + map2check_set_memcleanup(); + map2check_check_mem_endprog(); + assert_ok(); +} + +#[test] +fn endprog_keeps_an_earlier_violation() { + // #36: the end-of-program check must not overwrite the first violation. + let _guard = fresh(); + // SAFETY: fake address and C-string literals are valid. + unsafe { + map2check_map_malloc(addr(0x3000), 32); + map2check_check_load(std::ptr::null(), 9, 0, 4, c"first".as_ptr()); + } + map2check_check_mem_endprog(); + assert_violated(ViolatedProperty::MemsafetyDeref, 9, "first"); + map2check_set_memcleanup(); + map2check_check_mem_endprog(); + assert_violated(ViolatedProperty::MemsafetyDeref, 9, "first"); +} + +#[test] +fn null_is_valid_accepts_null_load() { + let _guard = fresh(); + map2check_init(); + map2check_set_null_is_valid(); + // SAFETY: null is never dereferenced. + unsafe { map2check_check_load(std::ptr::null(), 3, 0, 4, c"main".as_ptr()) }; + assert_ok(); +} + +/// Runs a leak check and a NULL load on a fresh analysis, reports the +/// (end-of-program property, null-load accepted) pair. +fn observed_options() -> (ViolatedProperty, bool) { + state::reset(); + // SAFETY: fake address is never dereferenced. + unsafe { map2check_map_malloc(addr(0x3000), 32) }; + map2check_check_mem_endprog(); + let leak = snapshot(|s| s.result.property); + state::reset(); + // SAFETY: null is never dereferenced. + unsafe { map2check_check_load(std::ptr::null(), 3, 0, 4, c"main".as_ptr()) }; + let null_ok = snapshot(|s| s.result.ok); + state::reset(); + (leak, null_ok) +} + +#[test] +fn options_survive_init_and_reset() { + let _guard = fresh(); + map2check_set_memcleanup(); + map2check_set_null_is_valid(); + let expected = (ViolatedProperty::MemsafetyMemcleanup, true); + assert_eq!(observed_options(), expected); + + map2check_init(); + assert_eq!(observed_options(), expected); + + map2check_reset(); + assert_eq!(observed_options(), expected); +} + +#[test] +fn options_default_off() { + let _guard = fresh(); + assert_eq!( + observed_options(), + (ViolatedProperty::MemsafetyMemtrack, false) + ); +} diff --git a/rust/src/state.rs b/rust/src/state.rs index 3689005..0eaff75 100644 --- a/rust/src/state.rs +++ b/rust/src/state.rs @@ -14,16 +14,61 @@ use crate::{ static ANALYSIS_STATE: LazyLock> = LazyLock::new(|| Mutex::new(AnalysisState::default())); +static CONFIG: Mutex = Mutex::new(Config::new()); + +/// Run configuration, kept apart from the analysis state: it changes only +/// through its setters, so `map2check_init()` and `map2check_reset()` never +/// clear it. Task 2.3 adds the analysis mode here. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct Config { + /// Set by `map2check_set_memcleanup()`: a leak at exit is reported as + /// `MemsafetyMemcleanup` instead of `MemsafetyMemtrack`. + pub memcleanup: bool, + /// Set by `map2check_set_null_is_valid()`: a NULL load is safe. + pub null_is_valid: bool, +} + +impl Config { + pub const fn new() -> Self { + Self { + memcleanup: false, + null_is_valid: false, + } + } +} + +impl Default for Config { + fn default() -> Self { + Self::new() + } +} + +/// Returns a copy of the current configuration. +pub fn config() -> Config { + *CONFIG + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) +} + +/// Changes the configuration. Only the option setters call this. +pub fn update_config(f: impl FnOnce(&mut Config)) { + f(&mut CONFIG + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner)); +} + +/// Restores the default configuration (tests only; the library never does). +#[cfg(test)] +pub(crate) fn reset_config() { + update_config(|c| *c = Config::new()); +} + #[derive(Debug, Default)] pub struct AnalysisState { pub result: AnalysisResult, pub nondets: Vec, pub memtrack: Vec, pub bbtrack: Vec, - /// Set by `map2check_set_null_is_valid()`; read by `map2check_check_load`. - pub null_is_valid: bool, - /// Set by `map2check_set_memcleanup()`; read by `map2check_check_mem_endprog`. - pub memcleanup_enabled: bool, pub legacy_nondet_logs: Vec>, pub legacy_bbtrack_logs: Vec>, } From 9a6dcd1490afbd169c9480dfc8696ae4a85ed016 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 21:15:29 -0400 Subject: [PATCH 18/63] test: align the C harness with the kept memory options (#10) --- tests/ffi_harness.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index 13f9942..3d0553a 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -323,10 +323,14 @@ static int test_result_and_control_flags(void) { map2check_reset(); map2check_map_malloc((void *)0x1000, 4); map2check_check_mem_endprog(); - if (get_current_property() != None) { + /* The pre-init round already set memcleanup and reset keeps options + * (#10), so the leak reads as memcleanup; the memtrack default is + * covered by the Rust tests (no API clears an option). */ + if (get_current_property() != MemsafetyMemcleanup) { return 3; } - map2check_set_memcleanup(); + map2check_reset(); + map2check_map_malloc((void *)0x1000, 4); map2check_check_mem_endprog(); if (get_current_property() != MemsafetyMemcleanup) { return 4; From 3e758114f0f0e58f213a210d4db548fc25d2295e Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 21:18:43 -0400 Subject: [PATCH 19/63] test: expect memtrack then memcleanup at exit in the C API test (#10) --- test/codeAPI_by_caller_test.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/test/codeAPI_by_caller_test.c b/test/codeAPI_by_caller_test.c index e66388d..7b8fb17 100644 --- a/test/codeAPI_by_caller_test.c +++ b/test/codeAPI_by_caller_test.c @@ -62,15 +62,18 @@ int main(void) { map2check_is_valid_assert(20, "main", 1); assert(get_current_property() == None); - /* map2check_check_mem_endprog is a no-op until map2check_set_memcleanup() - * is called (see rust/src/ffi.rs). The exact leak-detection heuristic - * itself is adjudicated in M6, so this test only exercises the on/off - * plumbing, not the heuristic's outcome. */ + /* The end-of-program leak check runs in both modes (SD-04): a leak is + * MemsafetyMemtrack by default and MemsafetyMemcleanup after + * map2check_set_memcleanup(), as in the develop C runtime. */ + map2check_reset(); map2check_map_malloc((void *)0x2000, 8); map2check_check_mem_endprog(); - assert(get_current_property() == None); + assert(get_current_property() == MemsafetyMemtrack); + map2check_reset(); map2check_set_memcleanup(); + map2check_map_malloc((void *)0x2000, 8); map2check_check_mem_endprog(); + assert(get_current_property() == MemsafetyMemcleanup); vcc_reset_meta_data(); map2check_success(); From a019f33ead29176df9de3cc9d40e70580237ea4c Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 21:32:18 -0400 Subject: [PATCH 20/63] feat: select analysis mode at runtime and report target reachability Add map2check_set_mode(AnalysisMode) and keep the mode in the run configuration, so init and reset keep it. Each analysis_init, analysis_destroy, analysis_is_program_correct and check gate of the develop AnalysisMode*.c files becomes a match in analysismode/mode.rs. With no mode set every check stays active, as before. Add map2check_reach_target, which records Reachability (develop TARGET-REACHED) and returns. Append Assert and Unknown to ViolatedProperty and report failed asserts as Assert. --- docs/api-deltas.md | 71 +++-- docs/cutover/abi-mapping.tsv | 4 +- docs/integration-guide.md | 4 +- docs/semantic-deltas.md | 73 ++++- rust/src/analysismode/assert.rs | 7 +- rust/src/analysismode/mod.rs | 1 + rust/src/analysismode/mode.rs | 485 ++++++++++++++++++++++++++++++++ rust/src/caller/mod.rs | 16 ++ rust/src/ffi.rs | 141 ++++++++-- rust/src/ffi_tests.rs | 273 +++++++++++++++++- rust/src/state.rs | 28 +- tests/ffi_harness.c | 42 +++ 12 files changed, 1080 insertions(+), 65 deletions(-) create mode 100644 rust/src/analysismode/mode.rs diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 32c05eb..41a37ab 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -43,17 +43,19 @@ build with `--features libfuzzer`. | Symbol | Rust declaration (`include/map2check.h`) | Legacy C library | Status | Notes | |---|---|---|---|---| -| `map2check_init` | `void map2check_init(void)` | `map2check_init` | same | Starts a fresh analysis (same as reset) and keeps the options (#10). Map2Check passes call `map2check_init(int)`; see the pass matrix. | -| `map2check_reset` | `void map2check_reset(void)` | — | new | Clears containers, result and step counter (#9). Keeps the options set by `map2check_set_memcleanup()` and `map2check_set_null_is_valid()`: they are configuration, not analysis state (#10). | +| `map2check_init` | `void map2check_init(void)` | `map2check_init` | same | Starts a fresh analysis through the `analysis_init` of the selected mode and keeps the options and the mode (#10, SD-14). Map2Check passes call `map2check_init(int)`; see the pass matrix. | +| `map2check_reset` | `void map2check_reset(void)` | — | new | Clears containers, result and step counter (#9). Keeps the options set by `map2check_set_memcleanup()` and `map2check_set_null_is_valid()` and the mode set by `map2check_set_mode()`: they are configuration, not analysis state (#10, SD-14). | | `map2check_success` | `void map2check_success(void)` | `map2check_success` | same | Prints the JSON report (SD-01). | -| `set_false_result` | `void set_false_result(int32_t prp, int32_t line_number, const char *function_name)` | `set_false_result` | same | `ViolatedProperty` keeps the C variant order (`caller_lib_result.h:22-31`); unknown values are ignored (SD-09). Ignored when a violation is already recorded (SD-03). | +| `set_false_result` | `void set_false_result(int32_t prp, int32_t line_number, const char *function_name)` | `set_false_result` | same | `ViolatedProperty` keeps the C variant order (`caller_lib_result.h:22-31`); the appended `Assert` (8) and `Unknown` (9) are accepted; unknown values are ignored (SD-09). Ignored when a violation is already recorded (SD-03). | | `get_current_property` | `enum ViolatedProperty get_current_property(void)` | `get_current_property` | same | | | `get_current_step` | `uint64_t get_current_step(void)` | `long get_current_step(void)` | changed | Returns `uint64_t` instead of `long`. | | `get_next_step` | `uint64_t get_next_step(void)` | `get_next_step()` / `next_current_step()` | same | Advances and returns the step counter. `next_current_step` is not exported; see its row. | | `print_all_containers_as_json` | `void print_all_containers_as_json(void)` | `print_all_containers_as_json` | changed | Report shape and whitespace differ (SD-01, SD-02; schema decision in #14). | | `vcc_reset_meta_data` | `void vcc_reset_meta_data(void)` | `vcc_reset_meta_data` | same | Host/test-only: the Map2Check passes never call it. It clears a recorded violation by design, bypassing the first-violation rule (SD-03). | -| `map2check_set_memcleanup` | `void map2check_set_memcleanup(void)` | `map2check_set_memcleanup` | same | Selects `MEMSAFETY_MEMCLEANUP` instead of `MEMSAFETY_MEMTRACK` for the end-of-program leak (SD-04). Kept by `map2check_init()` and `map2check_reset()`. | +| `map2check_set_memcleanup` | `void map2check_set_memcleanup(void)` | `map2check_set_memcleanup` | same | Selects `MEMSAFETY_MEMCLEANUP` instead of `MEMSAFETY_MEMTRACK` for the end-of-program leak (SD-04): an alias of the memcleanup mode for the leak tag, in every mode that checks leaks (SD-14). Kept by `map2check_init()` and `map2check_reset()`. | | `map2check_set_null_is_valid` | `void map2check_set_null_is_valid(void)` | `map2check_set_null_is_valid` | same | Makes a NULL load safe (SD-05). Kept by `map2check_init()` and `map2check_reset()`. | +| `map2check_set_mode` | `void map2check_set_mode(enum AnalysisMode mode)` | — | new | Selects the analysis mode at run time (`AnalysisMode_None` 0, `_Memtrack` 1, `_Memcleanup` 2, `_Overflow` 3, `_Assert` 4, `_Target` 5); develop selects it at link time. Called before `map2check_init()`; kept by `map2check_init()` and `map2check_reset()`. Never called: mode unset, every check active. See “Analysis modes” (SD-14). A value outside 0–5 is undefined behaviour. | +| `map2check_reach_target` | `void map2check_reach_target(const char *func, int32_t scope, int32_t line)` | — | new | Replaces the pass symbol `map2check_target_function`. Records `REACHABILITY` (develop `TARGET-REACHED`) at `line` in `func`, in every mode; `scope` is ignored. Does not end the process (SD-15). | | `map2check_save_nondet_log_int` | `non_det_log_t *map2check_save_nondet_log_int(int line, int scope, enum var_type_t type_var, int *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_uint` | `non_det_log_t *map2check_save_nondet_log_uint(int line, int scope, enum var_type_t type_var, unsigned int *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_long` | `non_det_log_t *map2check_save_nondet_log_long(int line, int scope, enum var_type_t type_var, long *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | @@ -69,24 +71,24 @@ build with `--features libfuzzer`. | `map2check_map_calloc` | `void map2check_map_calloc(const void *ptr_address, int32_t quantity, int32_t size)` | `map2check_map_calloc` | same | Negative sizes are ignored (SD-09). | | `map2check_map_free` | `void map2check_map_free(const char *var_name, const void *ptr_address, uint32_t scope, uint32_t line_number, const char *function_name)` | `map2check_map_free` | same | | | `map2check_map_store_pointer` | `void map2check_map_store_pointer(const void *var_address, const void *value, uint32_t scope, const char *var_name, int32_t line_number, const char *funct_name)` | `map2check_map_store_pointer` | same | | -| `map2check_is_valid_assert` | `void map2check_is_valid_assert(int32_t line_number, const char *function_name, int32_t expression)` | `map2check_is_valid_assert` | same | Prints the JSON report when the assertion fails. | -| `map2check_binop_add_int` | `void map2check_binop_add_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_add_int` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_sub_int` | `void map2check_binop_sub_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_sub_int` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_mul_int` | `void map2check_binop_mul_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_mul_int` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_div_int` | `void map2check_binop_div_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_div_int` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_neg_int` | `void map2check_binop_neg_int(int32_t param1, int32_t _param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_neg_int` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_shl_int` | `void map2check_binop_shl_int(int32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shl_int` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_shr_int` | `void map2check_binop_shr_int(int32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shr_int` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_add_unsigned` | `void map2check_binop_add_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_add_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_sub_unsigned` | `void map2check_binop_sub_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_sub_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_mul_unsigned` | `void map2check_binop_mul_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_mul_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_div_unsigned` | `void map2check_binop_div_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_div_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_binop_shr_unsigned` | `void map2check_binop_shr_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shr_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. | -| `map2check_check_load` | `void map2check_check_load(const void *ptr, int32_t line, uint32_t scope, int32_t size, const char *function_name)` | `map2check_check_load` | changed | Address-validity rule rewritten (plan R1; differential check in #15); since #11 a heap block is valid only while its latest alloc/free event is an allocation, as in `develop` (`AllocationLog.c:127-148`). The first violation is final; a later check never clears or replaces it (SD-03); NULL handling in SD-05. | -| `map2check_check_free` | `void map2check_check_free(const char *_name, const void *ptr, uint32_t _scope, uint32_t line, const char *function_name)` | `map2check_check_free` | same | `_name` and `_scope` are ignored. Untracked address is a violation (SD-06). | -| `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; records the release when the check passes, unlike `map2check_check_free`; untracked address is a violation (SD-06); see also SD-13. | -| `map2check_check_deref` | `void map2check_check_deref(const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_deref` | same | Name collides with the pass ABI `map2check_check_deref(i64, ptr)`; see the pass matrix (#21). | -| `map2check_check_mem_endprog` | `void map2check_check_mem_endprog(void)` | `map2check_check_mem_endprog` | changed | Leak algorithm rewritten; always runs, reporting memtrack or memcleanup by option (SD-04). | +| `map2check_is_valid_assert` | `void map2check_is_valid_assert(int32_t line_number, const char *function_name, int32_t expression)` | `map2check_is_valid_assert` | changed | A failed assertion reports `ASSERT` (before Task 2.3: `REACHABILITY`) and prints the JSON report. Active in unset and assert modes only (SD-14). | +| `map2check_binop_add_int` | `void map2check_binop_add_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_add_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_sub_int` | `void map2check_binop_sub_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_sub_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_mul_int` | `void map2check_binop_mul_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_mul_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_div_int` | `void map2check_binop_div_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_div_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_neg_int` | `void map2check_binop_neg_int(int32_t param1, int32_t _param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_neg_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_shl_int` | `void map2check_binop_shl_int(int32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shl_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_shr_int` | `void map2check_binop_shr_int(int32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shr_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_add_unsigned` | `void map2check_binop_add_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_add_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_sub_unsigned` | `void map2check_binop_sub_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_sub_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_mul_unsigned` | `void map2check_binop_mul_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_mul_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_div_unsigned` | `void map2check_binop_div_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_div_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_shr_unsigned` | `void map2check_binop_shr_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shr_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_check_load` | `void map2check_check_load(const void *ptr, int32_t line, uint32_t scope, int32_t size, const char *function_name)` | `map2check_check_load` | changed | Address-validity rule rewritten (plan R1; differential check in #15); since #11 a heap block is valid only while its latest alloc/free event is an allocation, as in `develop` (`AllocationLog.c:127-148`). The first violation is final; a later check never clears or replaces it (SD-03); NULL handling in SD-05. Active in unset and memtrack modes only (SD-14). | +| `map2check_check_free` | `void map2check_check_free(const char *_name, const void *ptr, uint32_t _scope, uint32_t line, const char *function_name)` | `map2check_check_free` | same | `_name` and `_scope` are ignored. Untracked address is a violation (SD-06). Active in unset and memtrack modes only (SD-14). | +| `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; records the release when the check passes, unlike `map2check_check_free`; untracked address is a violation (SD-06); see also SD-13. The check is active in unset and memtrack modes only; when inactive the release is still recorded (SD-14). | +| `map2check_check_deref` | `void map2check_check_deref(const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_deref` | same | Name collides with the pass ABI `map2check_check_deref(i64, ptr)`; see the pass matrix (#21). Active in unset and memtrack modes only (SD-14). | +| `map2check_check_mem_endprog` | `void map2check_check_mem_endprog(void)` | `map2check_check_mem_endprog` | changed | Leak algorithm rewritten. Runs `analysis_is_program_correct` of the mode: unset and memtrack report memtrack or memcleanup by option, memcleanup mode reports memcleanup, the other modes nothing (SD-04, SD-14). | | `LLVMFuzzerTestOneInput` | `int32_t LLVMFuzzerTestOneInput(const uint8_t *data, uintptr_t size)` | `LLVMFuzzerTestOneInput` | same | Feature `libfuzzer`. Resets the analysis state on every iteration (SD-11). | | `__VERIFIER_nondet_bool` | `bool __VERIFIER_nondet_bool(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | | `__VERIFIER_nondet_char` | `int8_t __VERIFIER_nondet_char(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | @@ -122,7 +124,7 @@ Map2Check. | Symbol | Signature expected by the passes | Status | Notes | |---|---|---|---| -| `map2check_init` | `map2check_init(int)` | changed | Core export takes `void`. | +| `map2check_init` | `map2check_init(int)` | changed | Core export takes `void`; the mode the frontend chose at link time comes from `map2check_set_mode()` (Task 2.3, SD-14). | | `map2check_check_deref` | `map2check_check_deref(i64, ptr)` | changed | Same name, incompatible core signature `(ptr, u32, u32, ptr)`. The plan (§2.1, D1) proposes renaming the core export to `map2check_check_deref_v2` inside the compat layer; decision in #21. | | `map2check_track_bb` | `(u32, ptr)` | pending | Core equivalent: `map2check_save_basic_block_log` (#21). | | `map2check_alloca` | `(ptr, ptr, i64, i64, i64, i64)` | pending | Core equivalent: `map2check_map_alloca` with `int32_t` sizes (#21). | @@ -135,7 +137,7 @@ Map2Check. | `map2check_load` | `(ptr, i64)` | pending | Core `map2check_check_load` takes 5 arguments (#21). | | `map2check_add_store_pointer` | 6 arguments | pending | Core `map2check_map_store_pointer` differs in order and types (#21). | | `map2check_function` | `(ptr, ptr)` | pending | Core equivalent: `map2check_map_funct_address` (#21). | -| `map2check_target_function` | — | pending | #21. | +| `map2check_target_function` | `(ptr, i32, i32)` | pending | Core equivalent: `map2check_reach_target`, same arguments; records and returns instead of ending the run (SD-15). The pass switches to it later in the cutover. | | `map2check_assume` | — | pending | #21. | | `map2check_assume_loop` | — | pending | #21. | | `map2check_crab_assume` | — | pending | #21. | @@ -156,6 +158,27 @@ named `map2check_property_*` that the Map2Check frontend parses. This library only prints the JSON report on stdout (SD-01). The file channel is tracked in #19. +## Analysis modes + +`map2check_set_mode()` replaces the link-time choice of +`AnalysisMode*.c` in the Map2Check frontend (`caller.cpp:575-612`). The +per-mode `match` is in `rust/src/analysismode/mode.rs`. Tracking calls +(`map2check_map_*`, nondet and basic-block logs) record in every mode. + +| Mode | Frontend mode | Checks that report | End of program (`map2check_check_mem_endprog`) | +|---|---|---|---| +| unset (`map2check_set_mode` never called) | — | all | leak → `MEMSAFETY_MEMTRACK`, or `MEMSAFETY_MEMCLEANUP` with the option | +| `AnalysisMode_None` (0) | `COVER_BRANCHES_MODE` | none | nothing | +| `AnalysisMode_Memtrack` (1) | `MEMTRACK_MODE` | load, deref, free, free resolved | leak → `MEMSAFETY_MEMTRACK`, or `MEMSAFETY_MEMCLEANUP` with the option | +| `AnalysisMode_Memcleanup` (2) | `MEMCLEANUP_MODE` | none (free resolved still records the release) | leak → `MEMSAFETY_MEMCLEANUP` | +| `AnalysisMode_Overflow` (3) | `OVERFLOW_MODE` | `map2check_binop_*` | nothing | +| `AnalysisMode_Assert` (4) | `ASSERT_MODE` | `map2check_is_valid_assert` (`ASSERT`) | nothing | +| `AnalysisMode_Target` (5) | `REACHABILITY_MODE` | none | nothing | + +`map2check_reach_target` and `set_false_result` record in every mode. +`ViolatedProperty` gained `Assert` (8) and `Unknown` (9), appended after +`None`; `Reachability` is the target-reached property. + ## Discontinued: `map2check_gen_data_fuzzer_pchar` / `_loff_t` / `_sector_t` **Status:** discontinued, not ported to `feature/rust`. diff --git a/docs/cutover/abi-mapping.tsv b/docs/cutover/abi-mapping.tsv index d791009..2e1aec3 100644 --- a/docs/cutover/abi-mapping.tsv +++ b/docs/cutover/abi-mapping.tsv @@ -1,8 +1,8 @@ c_symbol core_symbol owner_task notes -map2check_init map2check_set_mode+map2check_init 2.3 C: (i32 isSvComp) → núcleo: set_mode(proposto)+init(void); o arg int isSvComp é morto: Map2CheckFunctions.c:35 grava sv_comp e nenhum arquivo o lê +map2check_init map2check_set_mode+map2check_init 2.3 C: (i32 isSvComp) → núcleo: set_mode(AnalysisMode)+init(void) (Task 2.3; o modo vinha do .bc ligado, caller.cpp:575-612); o arg int isSvComp é morto: Map2CheckFunctions.c:35 grava sv_comp e nenhum arquivo o lê map2check_success map2check_success - C: () → núcleo: () (LibraryFunctions.hpp:48) map2check_track_bb map2check_save_basic_block_log - C: (i32 line,ptr fn) → núcleo: (line:i32,fn); mesma ordem (LibraryFunctions.hpp:54, ffi.rs:111); retorno do núcleo é bbtrack_log_t*, ignorado -map2check_target_function map2check_reach_target 2.3 C: (ptr fn,i32 scope,i32 line) (TargetPass.cpp:89); C runtime grava TARGET_REACHED e aborta (Map2CheckFunctions.c:61); núcleo proposto: (fn,scope,line) sem equivalente no header; C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (Map2CheckFunctions.c:64) +map2check_target_function map2check_reach_target 2.3 C: (ptr fn,i32 scope,i32 line) (TargetPass.cpp:89); C runtime grava TARGET_REACHED e aborta (Map2CheckFunctions.c:61-65); núcleo: map2check_reach_target(const char*,i32,i32) (Task 2.3) registra Reachability (=TARGET-REACHED) em qualquer modo, ignora scope e retorna; C encerra na primeira violação; núcleo registra e continua (SD-03/#36, SD-15); término do processo é a Task 2.4 map2check_assert map2check_is_valid_assert - C: (i32 cond,i32 line,ptr fn) (AssertPass.cpp:23) → núcleo: (line:i32,fn,expression:i32); ORDEM DIFERENTE: cond passa de primeiro a último; C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (map2check_error em AnalysisModeAssert.c:20); is_valid_assert imprime o JSON na hora sem encerrar, então um map2check_success posterior imprime um segundo relatório map2check_assume __VERIFIER_assume - C: (i32) (NonDetPass.cpp:94) → núcleo: (i32 condition); núcleo faz exit(0) se 0 map2check_assume_loop __VERIFIER_assume - C: (i1) (LoopPredAssumePass.cpp:28; o C runtime recebe char e faz cast para int) → núcleo: (i32); o passe deve emitir zext i1→i32 diff --git a/docs/integration-guide.md b/docs/integration-guide.md index 2af7513..4ed0090 100644 --- a/docs/integration-guide.md +++ b/docs/integration-guide.md @@ -181,8 +181,8 @@ print: ``` `property` takes one of `OVERFLOW`, `MEMSAFETY_FREE`, `MEMSAFETY_DEREF`, -`MEMSAFETY_MEMTRACK`, `MEMSAFETY_MEMCLEANUP`, `REACHABILITY` and -`CONCURRENCY`. Nondet `type` takes one of `INT`, `UINT`, `LONG`, `CHAR`, +`MEMSAFETY_MEMTRACK`, `MEMSAFETY_MEMCLEANUP`, `REACHABILITY`, +`CONCURRENCY`, `ASSERT` and `UNKNOWN`. Nondet `type` takes one of `INT`, `UINT`, `LONG`, `CHAR`, `FLOAT` and `DOUBLE`. The schema is not final: container and field names are still being reconciled with the existing consumer in #14 (SD-01). diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 778ff47..7354471 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -129,10 +129,12 @@ map2check_check_load((void *)0x1000, 2, 0, 4, "main"); **Rust:** `map2check_check_mem_endprog()` always runs the leak check. A dynamic allocation that was never freed reports `MEMSAFETY_MEMCLEANUP` after `map2check_set_memcleanup()` and `MEMSAFETY_MEMTRACK` otherwise. The option -is configuration: `map2check_init()` and `map2check_reset()` keep it. Until -the analysis mode is configurable (Task 2.3) the memcleanup option picks -between the memtrack and memcleanup verdicts. A violation recorded earlier -stays final (SD-03). +is configuration: `map2check_init()` and `map2check_reset()` keep it. Since +Task 2.3 the check runs `analysis_is_program_correct` of the analysis mode +(SD-14): with the mode unset or memtrack the option picks between the +memtrack and memcleanup verdicts, memcleanup mode always reports memcleanup, +and the other modes report no leak. A violation recorded earlier stays final +(SD-03). **Before #10:** the Rust function did nothing until `map2check_set_memcleanup()` had been called, so a leak in memtrack mode went @@ -387,3 +389,66 @@ map2check_init(); map2check_map_malloc((void *)0x1000, 8); map2check_check_free_resolved_address((void *)0x1000, 3, "main", 0); /* None; develop: FALSE_FREE */ ``` + +### SD-14 — The analysis mode is chosen at run time; unset keeps every check + +**Classification:** intentional +**Tests:** `rust/src/ffi_tests.rs::unset_mode_keeps_every_check_active`, `rust/src/ffi_tests.rs::mode_none_reports_nothing`, `rust/src/ffi_tests.rs::mode_memtrack_reports_memory_errors_only`, `rust/src/ffi_tests.rs::mode_memcleanup_reports_only_the_leak_as_memcleanup`, `rust/src/ffi_tests.rs::mode_overflow_reports_overflow_only`, `rust/src/ffi_tests.rs::mode_assert_reports_assert_only`, `rust/src/ffi_tests.rs::mode_target_reports_nothing_from_checks`, `rust/src/ffi_tests.rs::memcleanup_option_is_an_alias_of_memcleanup_mode_for_the_leak`, `rust/src/ffi_tests.rs::is_valid_assert_false_reports_assert`, `rust/src/analysismode/mode.rs::program_correct_memcleanup_reports_leak_as_memcleanup` + +**Rust:** `map2check_set_mode()` selects the mode before `map2check_init()`; +init and reset keep it. Each mode activates the checks that its +`AnalysisMode*.c` defines in develop; a check of another mode records +nothing. Tracking calls record in every mode, and an inactive +`map2check_check_free_resolved_address` still records the release. When +`map2check_set_mode()` is never called the mode is unset: every check is +active and the leak tag follows the memcleanup option, as before Task 2.3. +The memcleanup option is an alias of the memcleanup mode for the leak tag, +so memtrack mode plus the option reports `MEMSAFETY_MEMCLEANUP`. A failed +assert reports `ASSERT` instead of `REACHABILITY`. In memcleanup mode a leak +is any live allocation by the Rust leak rule; develop also requires +`is_memcleanup_error` (`ListLog.c:40-65`) to hold. + +**C runtime of Map2Check `develop`:** the mode is fixed at link time +(`modules/frontend/caller.cpp:575-612`): one of `AnalysisModeNone.c` +(cover-branches and reachability), `AnalysisModeMemtrack.c`, +`AnalysisModeMemcleanup.c`, `AnalysisModeOverflow.c` or +`AnalysisModeAssert.c` is linked, so calls of another mode are never emitted +and there is no unset mode. `map2check_assert` writes `ASSERT` +(`AnalysisModeAssert.c:18-23`, `PropertyGenerator.c:92-97`). The memcleanup +mode never fails a load, deref or free (`AnalysisModeMemcleanup.c:158`, +`:293`, `:166`, `:343`). + +**Reproduce:** + +```c +map2check_set_mode(AnalysisMode_None); +map2check_init(); +map2check_check_deref((void *)0xdead, 0, 1, "main"); /* None */ +map2check_set_mode(AnalysisMode_Memtrack); +map2check_init(); +map2check_check_deref((void *)0xdead, 0, 1, "main"); /* MemsafetyDeref */ +``` + +### SD-15 — Reaching the target records the violation and returns + +**Classification:** accepted limitation +**Tests:** `rust/src/ffi_tests.rs::map2check_reach_target_records_and_returns`, `rust/src/ffi_tests.rs::reach_target_keeps_the_first_violation`, `rust/src/ffi_tests.rs::reach_target_works_in_every_mode_and_accepts_null_name` + +**Rust:** `map2check_reach_target(func, scope, line)` records +`REACHABILITY` at `line` in `func` and returns. The first violation stays +final (SD-03). It works in every mode and ignores `scope`. Ending the run +with the result files belongs to Task 2.4 of the cutover plan; until then +the program keeps running after the target. + +**C runtime of Map2Check `develop`:** `map2check_target_function` writes +`TARGET-REACHED` and calls `map2check_error()`, which ends the process +(`Map2CheckFunctions.c:61-65`, `:72-75`, `:89-102`). It is part of +`Map2CheckFunctions.c`, linked in every mode. + +**Reproduce:** + +```c +map2check_set_mode(AnalysisMode_Target); +map2check_init(); +map2check_reach_target("reach_error", 0, 12); /* Reachability, line 12; returns */ +``` diff --git a/rust/src/analysismode/assert.rs b/rust/src/analysismode/assert.rs index 8f79b82..9fd6d33 100644 --- a/rust/src/analysismode/assert.rs +++ b/rust/src/analysismode/assert.rs @@ -17,8 +17,9 @@ pub struct AssertChecker { impl VccChecker for AssertChecker { fn check(&self, _ctx: &VccContext) -> Result { if self.expression == 0 { + // develop: map2check_assert writes ASSERT (AnalysisModeAssert.c:18-23). Ok(VccOutcome::Violated { - property: ViolatedProperty::Reachability, + property: ViolatedProperty::Assert, }) } else { Ok(VccOutcome::Safe) @@ -36,13 +37,13 @@ mod tests { } #[test] - fn false_expression_yields_reachability_violation() { + fn false_expression_yields_assert_violation() { let checker = AssertChecker { expression: 0 }; let outcome = checker.check(&ctx()).unwrap(); assert_eq!( outcome, VccOutcome::Violated { - property: ViolatedProperty::Reachability + property: ViolatedProperty::Assert } ); } diff --git a/rust/src/analysismode/mod.rs b/rust/src/analysismode/mod.rs index 3f0b09e..dc93805 100644 --- a/rust/src/analysismode/mod.rs +++ b/rust/src/analysismode/mod.rs @@ -7,6 +7,7 @@ use crate::error::Map2CheckError; pub mod assert; pub mod memory; +pub mod mode; pub mod overflow; /// Context passed to every VCC checker — carries the source location. diff --git a/rust/src/analysismode/mode.rs b/rust/src/analysismode/mode.rs new file mode 100644 index 0000000..6b737cd --- /dev/null +++ b/rust/src/analysismode/mode.rs @@ -0,0 +1,485 @@ +// SPDX-License-Identifier: GPL-3.0-only +// Runtime selection of the analysis mode (Task 2.3 of the cutover plan). +#![deny(unsafe_code)] +// +// In the C runtime of Map2Check `develop` the analysis mode is chosen at link +// time: the frontend links exactly one `AnalysisMode*.c` bitcode per run +// (`modules/frontend/caller.cpp:575-612`). Each of those files defines +// `analysis_init`, `analysis_destroy`, `analysis_is_program_correct` and the +// check functions of its mode. This library is a single artifact, so the mode +// is chosen at run time by `map2check_set_mode()` and every one of those +// functions becomes a `match` on the mode below. +// +// The mode is optional. When `map2check_set_mode()` was never called it is +// unset (`None` here) and the library behaves as it did before Task 2.3: every +// check is active and the end-of-program leak tag follows the memcleanup +// option. + +use crate::{ + caller::ViolatedProperty, + memtrack, + state::{AnalysisState, Config}, +}; + +/// Analysis mode selected by the instrumented program before +/// `map2check_init()`. The numeric values are part of the ABI: the LLVM +/// passes emit them as constants. +/// +/// Frontend mapping (`modules/frontend/caller.hpp:21-37`, +/// `caller.cpp:575-612`): `MEMTRACK_MODE` → `Memtrack`, `MEMCLEANUP_MODE` → +/// `Memcleanup`, `OVERFLOW_MODE` → `Overflow`, `ASSERT_MODE` → `Assert`, +/// `REACHABILITY_MODE` → `Target`, `COVER_BRANCHES_MODE` → `None`. Both +/// `REACHABILITY_MODE` and `COVER_BRANCHES_MODE` link `AnalysisModeNone.c`. +/// +/// cbindgen:prefix-with-name +#[repr(C)] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum AnalysisMode { + None = 0, + Memtrack = 1, + Memcleanup = 2, + Overflow = 3, + Assert = 4, + Target = 5, +} + +/// A check whose activity depends on the analysis mode. Tracking calls +/// (`map2check_map_*`, nondet and basic-block logs) are not checks: they +/// record in every mode. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Check { + /// `map2check_check_load`. + Load, + /// `map2check_check_deref`. + Deref, + /// `map2check_check_free`. + Free, + /// `map2check_check_free_resolved_address` (the check only; the release + /// is recorded whenever the check is inactive or passes). + FreeResolved, + /// Every `map2check_binop_*` export. + Overflow, + /// `map2check_is_valid_assert`. + Assert, +} + +/// `analysis_init()`: starts a fresh analysis. Called by `map2check_init()` +/// (develop: `map2check_init` calls `analysis_init`, +/// `Map2CheckFunctions.c:29-40`). +/// +/// Every arm clears the whole state. In develop only the memory modes have +/// containers to create; the other modes have none, and the step counter and +/// basic-block log are reset by `map2check_init` itself (`:30-32`). Clearing +/// everything in every mode gives the same observable start: empty containers, +/// step 0 and no verdict. +pub fn analysis_init(mode: Option, state: &mut AnalysisState) { + match mode { + // Unset: the pre-Task-2.3 `map2check_init()`, which cleared everything. + None => state.reset(), + // AnalysisModeNone.c:11-13 — "DO NOTHING". + Some(AnalysisMode::None) => state.reset(), + // AnalysisModeMemtrack.c:125-130 — new heap/allocation/list logs, + // ERROR_DEREF = FALSE. + Some(AnalysisMode::Memtrack) => state.reset(), + // AnalysisModeMemcleanup.c:125-130 — same as memtrack. + Some(AnalysisMode::Memcleanup) => state.reset(), + // AnalysisModeOverflow.c:279 — empty. + Some(AnalysisMode::Overflow) => state.reset(), + // AnalysisModeAssert.c:29-31 — "DO NOTHING". + Some(AnalysisMode::Assert) => state.reset(), + // REACHABILITY_MODE links AnalysisModeNone.c (caller.cpp:600-605). + Some(AnalysisMode::Target) => state.reset(), + } +} + +/// `analysis_destroy()`: releases the containers of the mode. The verdict, +/// the step counter and the nondet/basic-block logs are kept (develop frees +/// the basic-block log in `map2check_destroy` itself, +/// `Map2CheckFunctions.c:77-87`). Ending the run is Task 2.4, which calls +/// this after the end-of-program verdict. +pub fn analysis_destroy(mode: Option, state: &mut AnalysisState) { + match mode { + // Unset: every check is active, so the memory log is in use. + None => state.memtrack.clear(), + // AnalysisModeMemtrack.c:132-136 — frees heap, list and allocation logs. + Some(AnalysisMode::Memtrack) => state.memtrack.clear(), + // AnalysisModeMemcleanup.c:132-136 — same as memtrack. + Some(AnalysisMode::Memcleanup) => state.memtrack.clear(), + // AnalysisModeNone.c:15-17 — "DO NOTHING". + Some(AnalysisMode::None) => {} + // AnalysisModeOverflow.c:281 — empty. + Some(AnalysisMode::Overflow) => {} + // AnalysisModeAssert.c:33-35 — "DO NOTHING". + Some(AnalysisMode::Assert) => {} + // REACHABILITY_MODE links AnalysisModeNone.c (caller.cpp:600-605). + Some(AnalysisMode::Target) => {} + } +} + +/// `analysis_is_program_correct()`: the end-of-program verdict. Returns the +/// violated property, or `None` when the program is correct. Develop calls it +/// from `map2check_exit` when no error was recorded +/// (`Map2CheckFunctions.c:89-92`); here `map2check_check_mem_endprog()` does. +/// +/// The memcleanup option is an alias of the memcleanup mode for the leak tag: +/// in every mode that checks leaks it selects `MemsafetyMemcleanup`. +pub fn analysis_is_program_correct( + config: Config, + state: &AnalysisState, +) -> Option { + let leak = |property| memtrack::has_memcleanup_error(&state.memtrack).then_some(property); + let memtrack_tag = if config.memcleanup { + ViolatedProperty::MemsafetyMemcleanup + } else { + ViolatedProperty::MemsafetyMemtrack + }; + match config.mode { + // Unset: the pre-Task-2.3 leak check, tagged by the option (SD-04). + None => leak(memtrack_tag), + // AnalysisModeMemtrack.c:138-146 — a live allocation is FALSE_MEMTRACK. + Some(AnalysisMode::Memtrack) => leak(memtrack_tag), + // AnalysisModeMemcleanup.c:138-148 — a live allocation is + // FALSE_MEMCLEANUP (develop also asks `is_memcleanup_error`, + // ListLog.c:40-65; Rust uses its own leak rule, see SD-14). + Some(AnalysisMode::Memcleanup) => leak(ViolatedProperty::MemsafetyMemcleanup), + // AnalysisModeNone.c:19 — TRUE. + Some(AnalysisMode::None) => None, + // AnalysisModeOverflow.c:283 — TRUE. + Some(AnalysisMode::Overflow) => None, + // AnalysisModeAssert.c:37 — TRUE. + Some(AnalysisMode::Assert) => None, + // REACHABILITY_MODE links AnalysisModeNone.c: TRUE. + Some(AnalysisMode::Target) => None, + } +} + +/// Whether `check` reports violations in `mode`. An inactive check records +/// nothing, as when develop links a mode whose file has no such check (the +/// pass of another mode never emits the call) or whose check never fails. +pub fn check_enabled(mode: Option, check: Check) -> bool { + match mode { + // Unset: every check is active, as before Task 2.3. + None => true, + // AnalysisModeNone.c defines no check. + Some(AnalysisMode::None) => false, + // AnalysisModeMemtrack.c: map2check_load (:156-162), check_deref + // (:297-303), map2check_free (:339-366), free_resolved_address + // (:164-183). No overflow or assert check is linked. + Some(AnalysisMode::Memtrack) => matches!( + check, + Check::Load | Check::Deref | Check::Free | Check::FreeResolved + ), + // AnalysisModeMemcleanup.c: map2check_load is empty (:158), + // check_deref is empty (:293), map2check_free and + // free_resolved_address use `error = FALSE` (:343, :166). Only the + // end-of-program leak is checked. + Some(AnalysisMode::Memcleanup) => false, + // AnalysisModeOverflow.c:185-276 — the binop checks only. + Some(AnalysisMode::Overflow) => check == Check::Overflow, + // AnalysisModeAssert.c:18-23 — map2check_assert only. + Some(AnalysisMode::Assert) => check == Check::Assert, + // REACHABILITY_MODE links AnalysisModeNone.c; the target call is + // map2check_target_function, outside the mode (Map2CheckFunctions.c:61). + Some(AnalysisMode::Target) => false, + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::memtrack::MemTrackEntry; + + const ALL_CHECKS: [Check; 6] = [ + Check::Load, + Check::Deref, + Check::Free, + Check::FreeResolved, + Check::Overflow, + Check::Assert, + ]; + + fn malloc(addr: usize) -> MemTrackEntry { + let mut e = MemTrackEntry::new(1, 0, 0, addr, addr, false, false, "", "", 16, 1, false); + e.set_malloc(); + e + } + + /// A state with one leaked heap block, a recorded violation and a step. + fn dirty_state() -> AnalysisState { + let mut s = AnalysisState::default(); + s.memtrack.push(malloc(0x1000)); + s.set_false(ViolatedProperty::Overflow, 3, "f"); + s.next_step(); + s + } + + fn leaked_state() -> AnalysisState { + let mut s = AnalysisState::default(); + s.memtrack.push(malloc(0x1000)); + s + } + + fn config(mode: Option, memcleanup: bool) -> Config { + Config { + mode, + memcleanup, + ..Config::new() + } + } + + fn enabled(mode: Option) -> Vec { + ALL_CHECKS + .into_iter() + .filter(|c| check_enabled(mode, *c)) + .collect() + } + + fn assert_fresh(s: &AnalysisState) { + assert!(s.result.ok); + assert_eq!(s.result.property, ViolatedProperty::None); + assert_eq!(s.current_step(), 0); + assert!(s.memtrack.is_empty() && s.nondets.is_empty() && s.bbtrack.is_empty()); + } + + // ── AnalysisMode ABI ──────────────────────────────────────────────────── + + #[test] + fn analysis_mode_discriminants_are_the_abi_values() { + assert_eq!(AnalysisMode::None as i32, 0); + assert_eq!(AnalysisMode::Memtrack as i32, 1); + assert_eq!(AnalysisMode::Memcleanup as i32, 2); + assert_eq!(AnalysisMode::Overflow as i32, 3); + assert_eq!(AnalysisMode::Assert as i32, 4); + assert_eq!(AnalysisMode::Target as i32, 5); + } + + // ── analysis_init: one test per arm ───────────────────────────────────── + + #[test] + fn init_unset_starts_fresh() { + let mut s = dirty_state(); + analysis_init(None, &mut s); + assert_fresh(&s); + } + + #[test] + fn init_none_starts_fresh() { + let mut s = dirty_state(); + analysis_init(Some(AnalysisMode::None), &mut s); + assert_fresh(&s); + } + + #[test] + fn init_memtrack_starts_fresh() { + let mut s = dirty_state(); + analysis_init(Some(AnalysisMode::Memtrack), &mut s); + assert_fresh(&s); + } + + #[test] + fn init_memcleanup_starts_fresh() { + let mut s = dirty_state(); + analysis_init(Some(AnalysisMode::Memcleanup), &mut s); + assert_fresh(&s); + } + + #[test] + fn init_overflow_starts_fresh() { + let mut s = dirty_state(); + analysis_init(Some(AnalysisMode::Overflow), &mut s); + assert_fresh(&s); + } + + #[test] + fn init_assert_starts_fresh() { + let mut s = dirty_state(); + analysis_init(Some(AnalysisMode::Assert), &mut s); + assert_fresh(&s); + } + + #[test] + fn init_target_starts_fresh() { + let mut s = dirty_state(); + analysis_init(Some(AnalysisMode::Target), &mut s); + assert_fresh(&s); + } + + // ── analysis_destroy: one test per arm ────────────────────────────────── + + #[test] + fn destroy_unset_releases_memory_containers() { + let mut s = dirty_state(); + analysis_destroy(None, &mut s); + assert!(s.memtrack.is_empty()); + assert!(!s.result.ok, "destroy keeps the verdict"); + } + + #[test] + fn destroy_none_keeps_everything() { + let mut s = dirty_state(); + analysis_destroy(Some(AnalysisMode::None), &mut s); + assert_eq!(s.memtrack.len(), 1); + assert!(!s.result.ok); + } + + #[test] + fn destroy_memtrack_releases_memory_containers() { + let mut s = dirty_state(); + analysis_destroy(Some(AnalysisMode::Memtrack), &mut s); + assert!(s.memtrack.is_empty()); + assert!(!s.result.ok); + } + + #[test] + fn destroy_memcleanup_releases_memory_containers() { + let mut s = dirty_state(); + analysis_destroy(Some(AnalysisMode::Memcleanup), &mut s); + assert!(s.memtrack.is_empty()); + assert!(!s.result.ok); + } + + #[test] + fn destroy_overflow_keeps_everything() { + let mut s = dirty_state(); + analysis_destroy(Some(AnalysisMode::Overflow), &mut s); + assert_eq!(s.memtrack.len(), 1); + } + + #[test] + fn destroy_assert_keeps_everything() { + let mut s = dirty_state(); + analysis_destroy(Some(AnalysisMode::Assert), &mut s); + assert_eq!(s.memtrack.len(), 1); + } + + #[test] + fn destroy_target_keeps_everything() { + let mut s = dirty_state(); + analysis_destroy(Some(AnalysisMode::Target), &mut s); + assert_eq!(s.memtrack.len(), 1); + } + + // ── analysis_is_program_correct: one test per arm ─────────────────────── + + #[test] + fn program_correct_unset_reports_leak_by_option() { + let s = leaked_state(); + assert_eq!( + analysis_is_program_correct(config(None, false), &s), + Some(ViolatedProperty::MemsafetyMemtrack) + ); + assert_eq!( + analysis_is_program_correct(config(None, true), &s), + Some(ViolatedProperty::MemsafetyMemcleanup) + ); + assert_eq!( + analysis_is_program_correct(config(None, false), &AnalysisState::default()), + None + ); + } + + #[test] + fn program_correct_none_ignores_leak() { + let s = leaked_state(); + for memcleanup in [false, true] { + let c = config(Some(AnalysisMode::None), memcleanup); + assert_eq!(analysis_is_program_correct(c, &s), None); + } + } + + #[test] + fn program_correct_memtrack_reports_leak_as_memtrack() { + let s = leaked_state(); + let c = config(Some(AnalysisMode::Memtrack), false); + assert_eq!( + analysis_is_program_correct(c, &s), + Some(ViolatedProperty::MemsafetyMemtrack) + ); + assert_eq!( + analysis_is_program_correct(c, &AnalysisState::default()), + None + ); + } + + #[test] + fn program_correct_memtrack_with_memcleanup_option_reports_memcleanup() { + // The option is an alias of the memcleanup mode for the leak tag. + let c = config(Some(AnalysisMode::Memtrack), true); + assert_eq!( + analysis_is_program_correct(c, &leaked_state()), + Some(ViolatedProperty::MemsafetyMemcleanup) + ); + } + + #[test] + fn program_correct_memcleanup_reports_leak_as_memcleanup() { + let s = leaked_state(); + let c = config(Some(AnalysisMode::Memcleanup), false); + assert_eq!( + analysis_is_program_correct(c, &s), + Some(ViolatedProperty::MemsafetyMemcleanup) + ); + assert_eq!( + analysis_is_program_correct(c, &AnalysisState::default()), + None + ); + } + + #[test] + fn program_correct_overflow_ignores_leak() { + let c = config(Some(AnalysisMode::Overflow), true); + assert_eq!(analysis_is_program_correct(c, &leaked_state()), None); + } + + #[test] + fn program_correct_assert_ignores_leak() { + let c = config(Some(AnalysisMode::Assert), true); + assert_eq!(analysis_is_program_correct(c, &leaked_state()), None); + } + + #[test] + fn program_correct_target_ignores_leak() { + let c = config(Some(AnalysisMode::Target), true); + assert_eq!(analysis_is_program_correct(c, &leaked_state()), None); + } + + // ── check_enabled: one test per arm ───────────────────────────────────── + + #[test] + fn checks_unset_all_active() { + assert_eq!(enabled(None), ALL_CHECKS.to_vec()); + } + + #[test] + fn checks_none_all_inactive() { + assert_eq!(enabled(Some(AnalysisMode::None)), vec![]); + } + + #[test] + fn checks_memtrack_memory_only() { + assert_eq!( + enabled(Some(AnalysisMode::Memtrack)), + vec![Check::Load, Check::Deref, Check::Free, Check::FreeResolved] + ); + } + + #[test] + fn checks_memcleanup_none_active() { + assert_eq!(enabled(Some(AnalysisMode::Memcleanup)), vec![]); + } + + #[test] + fn checks_overflow_overflow_only() { + assert_eq!(enabled(Some(AnalysisMode::Overflow)), vec![Check::Overflow]); + } + + #[test] + fn checks_assert_assert_only() { + assert_eq!(enabled(Some(AnalysisMode::Assert)), vec![Check::Assert]); + } + + #[test] + fn checks_target_all_inactive() { + assert_eq!(enabled(Some(AnalysisMode::Target)), vec![]); + } +} diff --git a/rust/src/caller/mod.rs b/rust/src/caller/mod.rs index 9ce3350..df7a27f 100644 --- a/rust/src/caller/mod.rs +++ b/rust/src/caller/mod.rs @@ -17,6 +17,11 @@ pub enum ViolatedProperty { Concurrency, #[default] None, + /// A failed `map2check_is_valid_assert` (`ASSERT` in `PropertyGenerator.c`). + /// `Reachability` is the target-reached property (`TARGET_REACHED`). + Assert, + /// The analysis could not decide (`UNKNOWN` in `PropertyGenerator.c`). + Unknown, } impl ViolatedProperty { @@ -30,6 +35,8 @@ impl ViolatedProperty { Self::Reachability => "REACHABILITY", Self::Concurrency => "CONCURRENCY", Self::None => "NONE", + Self::Assert => "ASSERT", + Self::Unknown => "UNKNOWN", } } } @@ -75,6 +82,13 @@ mod tests { assert_eq!(ViolatedProperty::None.as_str(), "NONE"); } + #[test] + fn new_variants_are_appended_after_the_original_ones() { + assert_eq!(ViolatedProperty::None as i32, 7); + assert_eq!(ViolatedProperty::Assert as i32, 8); + assert_eq!(ViolatedProperty::Unknown as i32, 9); + } + #[test] fn violated_property_as_str_covers_every_variant() { let cases = [ @@ -89,6 +103,8 @@ mod tests { (ViolatedProperty::Reachability, "REACHABILITY"), (ViolatedProperty::Concurrency, "CONCURRENCY"), (ViolatedProperty::None, "NONE"), + (ViolatedProperty::Assert, "ASSERT"), + (ViolatedProperty::Unknown, "UNKNOWN"), ]; for (property, name) in cases { assert_eq!(property.as_str(), name); diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index ee48db3..c4d6209 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -12,7 +12,9 @@ use crate::analysismode::memory::DerefChecker; use crate::analysismode::memory::FreeChecker; use crate::analysismode::memory::FreeResolvedChecker; use crate::analysismode::memory::LoadChecker; -use crate::analysismode::memory::MemCleanupChecker; +use crate::analysismode::mode; +use crate::analysismode::mode::AnalysisMode; +use crate::analysismode::mode::Check; use crate::analysismode::overflow::AddI32; use crate::analysismode::overflow::AddU32; use crate::analysismode::overflow::DivI32; @@ -77,6 +79,11 @@ fn run_vcc_check(checker: impl VccChecker, ctx: &VccContext, line: u32, fname: & } } +/// Whether `check` reports violations in the selected analysis mode. +fn check_active(check: Check) -> bool { + mode::check_enabled(state::config().mode, check) +} + fn violated_property_from_raw(raw: i32) -> Option { match raw { 0 => Some(ViolatedProperty::Overflow), @@ -87,6 +94,8 @@ fn violated_property_from_raw(raw: i32) -> Option { 5 => Some(ViolatedProperty::Reachability), 6 => Some(ViolatedProperty::Concurrency), 7 => Some(ViolatedProperty::None), + 8 => Some(ViolatedProperty::Assert), + 9 => Some(ViolatedProperty::Unknown), _ => None, } } @@ -261,6 +270,69 @@ pub extern "C" fn map2check_set_null_is_valid() { ); } +/// Selects the analysis mode, as the frontend of Map2Check `develop` does by +/// linking one `AnalysisMode*.c` per run (`caller.cpp:575-612`). The +/// instrumented program calls it before `map2check_init()`. The mode is +/// configuration: `map2check_init()` and `map2check_reset()` keep it. Until +/// it is called the mode is unset and every check is active. Per-mode effects +/// are in `rust/src/analysismode/mode.rs`. +/// +/// `mode` must be one of the `AnalysisMode` values (0 to 5); any other value +/// is undefined behaviour, as for any C enum passed to Rust. +#[no_mangle] +pub extern "C" fn map2check_set_mode(mode: AnalysisMode) { + ffi_guard( + || { + state::update_config(|c| c.mode = Some(mode)); + }, + (), + ); +} + +/// Records that the target function was reached, without ending the process. +/// Same property as `map2check_target_function` of the `develop` C runtime +/// (`TARGET_REACHED`, `Map2CheckFunctions.c:61-65`), which then ends the run; +/// here the first violation stays final (SD-03) and the caller keeps running. +/// +/// # Safety +/// `func` must be null or a valid null-terminated C string. +pub(crate) unsafe fn mark_target_reached( + func: *const std::os::raw::c_char, + _scope: i32, + line: i32, +) { + // SAFETY: caller guarantees func is a valid C string or null. + let fname = unsafe { c_str(func) }; + if let Err(e) = state::with_state(|s| { + s.set_false(ViolatedProperty::Reachability, line as u32, fname); + Ok(()) + }) { + log_error(&e); + } +} + +/// Called by the target pass when the target function (e.g. `reach_error`) +/// is reached. Records `Reachability` (develop's `TARGET-REACHED`) at `line` +/// in `func`, in every mode, and returns: ending the run is Task 2.4 (SD-15). +/// `scope` is accepted for ABI compatibility and ignored, as in develop. +/// +/// # Safety +/// `func` must be null or a valid null-terminated C string. +#[no_mangle] +pub unsafe extern "C" fn map2check_reach_target( + func: *const std::os::raw::c_char, + scope: i32, + line: i32, +) { + ffi_guard( + || { + // SAFETY: caller guarantees func is a valid C string or null. + unsafe { mark_target_reached(func, scope, line) } + }, + (), + ); +} + // ── NonDet tracking ─────────────────────────────────────────────────────────── #[repr(C)] @@ -768,6 +840,9 @@ pub unsafe extern "C" fn map2check_is_valid_assert( ) { ffi_guard( || { + if !check_active(Check::Assert) { + return; + } // SAFETY: caller guarantees function_name is a valid C string or null. let fname = unsafe { c_str(function_name) }; let ctx = VccContext::new(line_number as u32, 0, fname); @@ -805,6 +880,9 @@ macro_rules! overflow_ffi_binop { ) { ffi_guard( || { + if !check_active(Check::Overflow) { + return; + } // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; let ctx = VccContext::new(line, scope, fname); @@ -838,6 +916,9 @@ macro_rules! overflow_ffi_shift { ) { ffi_guard( || { + if !check_active(Check::Overflow) { + return; + } // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; let ctx = VccContext::new(line, scope, fname); @@ -869,6 +950,9 @@ pub unsafe extern "C" fn map2check_binop_neg_int( ) { ffi_guard( || { + if !check_active(Check::Overflow) { + return; + } // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; let ctx = VccContext::new(line, scope, fname); @@ -906,6 +990,9 @@ pub unsafe extern "C" fn map2check_check_load( ) { ffi_guard( || { + if !check_active(Check::Load) { + return; + } // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; if size < 0 { @@ -948,6 +1035,9 @@ pub unsafe extern "C" fn map2check_check_free( ) { ffi_guard( || { + if !check_active(Check::Free) { + return; + } // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; let ctx = VccContext::new(line, 0, fname); @@ -993,14 +1083,17 @@ pub unsafe extern "C" fn map2check_check_free_resolved_address( return; } let ctx = VccContext::new(line, 0, fname); - let passed = run_vcc_check( - FreeResolvedChecker { - address: ptr as usize, - }, - &ctx, - line, - fname, - ); + // An inactive check passes: the memcleanup mode never fails a + // free but still records the release (AnalysisModeMemcleanup.c:160-179). + let passed = !check_active(Check::FreeResolved) + || run_vcc_check( + FreeResolvedChecker { + address: ptr as usize, + }, + &ctx, + line, + fname, + ); if !passed || ptr.is_null() { return; } @@ -1034,6 +1127,9 @@ pub unsafe extern "C" fn map2check_check_deref( ) { ffi_guard( || { + if !check_active(Check::Deref) { + return; + } // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; let ctx = VccContext::new(line, scope, fname); @@ -1050,22 +1146,25 @@ pub unsafe extern "C" fn map2check_check_deref( ); } -/// Checks for memory leaks at end of program, in both modes (SD-04). A leak is -/// `MemsafetyMemcleanup` after `map2check_set_memcleanup()`, otherwise -/// `MemsafetyMemtrack`, as in the `develop` C runtime. +/// End-of-program verdict: runs `analysis_is_program_correct` of the selected +/// mode (`rust/src/analysismode/mode.rs`). Unset mode and memtrack report a +/// leak as `MemsafetyMemtrack`, or `MemsafetyMemcleanup` after +/// `map2check_set_memcleanup()`; memcleanup mode reports it as +/// `MemsafetyMemcleanup`; the other modes report nothing (SD-04). A violation +/// recorded earlier stays final (SD-03). #[no_mangle] pub extern "C" fn map2check_check_mem_endprog() { ffi_guard( || { - let ctx = VccContext::new(0, 0, "end_of_program"); - run_vcc_check( - MemCleanupChecker { - memcleanup: state::config().memcleanup, - }, - &ctx, - 0, - "end_of_program", - ); + let config = state::config(); + if let Err(e) = state::with_state(|s| { + if let Some(property) = mode::analysis_is_program_correct(config, s) { + s.set_false(property, 0, "end_of_program"); + } + Ok(()) + }) { + log_error(&e); + } }, (), ); diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 86181c6..7d1f412 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -548,11 +548,13 @@ fn is_valid_assert_true_keeps_result_ok() { } #[test] -fn is_valid_assert_false_reports_reachability() { +fn is_valid_assert_false_reports_assert() { + // develop: map2check_assert writes ASSERT (AnalysisModeAssert.c:18-23), + // not TARGET-REACHED (Task 2.3). let _guard = fresh(); // SAFETY: C-string literal is valid and null-terminated. unsafe { map2check_is_valid_assert(5, c"main".as_ptr(), 0) }; - assert_violated(ViolatedProperty::Reachability, 5, "main"); + assert_violated(ViolatedProperty::Assert, 5, "main"); } // ── Analysis mode: overflow ───────────────────────────────────────────────── @@ -1243,3 +1245,270 @@ fn options_default_off() { (ViolatedProperty::MemsafetyMemtrack, false) ); } + +// ── Analysis mode selection and target reachability (Task 2.3) ───────────── + +const MODES: [AnalysisMode; 6] = [ + AnalysisMode::None, + AnalysisMode::Memtrack, + AnalysisMode::Memcleanup, + AnalysisMode::Overflow, + AnalysisMode::Assert, + AnalysisMode::Target, +]; + +/// Starts an analysis in `mode`, as the instrumented program will. +fn start(mode: AnalysisMode) { + state::reset(); + map2check_set_mode(mode); + map2check_init(); +} + +/// Runs one violating call of each check family on a fresh analysis in +/// `mode` and returns the property each one recorded (`None` = no report). +fn reports_in(mode: AnalysisMode) -> [ViolatedProperty; 7] { + let mut out = [ViolatedProperty::None; 7]; + let runs: [&dyn Fn(); 7] = [ + // SAFETY (all closures): C-string literals are valid; fake addresses + // are never dereferenced. + &|| unsafe { map2check_check_load(addr(0xdead), 1, 0, 4, c"main".as_ptr()) }, + &|| unsafe { map2check_check_deref(addr(0xdead), 0, 1, c"main".as_ptr()) }, + &|| unsafe { map2check_check_free(c"p".as_ptr(), addr(0xdead), 0, 1, c"main".as_ptr()) }, + &|| unsafe { map2check_check_free_resolved_address(addr(0xdead), 1, c"main".as_ptr(), 0) }, + &|| unsafe { map2check_binop_add_int(i32::MAX, 1, 1, 0, c"main".as_ptr()) }, + &|| unsafe { map2check_is_valid_assert(1, c"main".as_ptr(), 0) }, + &|| { + // SAFETY: fake address is never dereferenced. + unsafe { map2check_map_malloc(addr(0x3000), 8) }; + map2check_check_mem_endprog(); + }, + ]; + for (slot, run) in out.iter_mut().zip(runs) { + start(mode); + run(); + *slot = snapshot(|s| s.result.property); + } + out +} + +use ViolatedProperty as P; + +#[test] +fn mode_none_never_reports_memory_errors() { + let _guard = fresh(); + map2check_set_mode(AnalysisMode::None); + map2check_init(); + // SAFETY: C-string literal is valid; fake address is never dereferenced. + unsafe { map2check_check_deref(addr(0xdead), 0, 1, c"main".as_ptr()) }; + assert_ok(); +} + +#[test] +fn mode_memtrack_reports_invalid_deref() { + let _guard = fresh(); + map2check_set_mode(AnalysisMode::Memtrack); + map2check_init(); + // SAFETY: C-string literal is valid; fake address is never dereferenced. + unsafe { map2check_check_deref(addr(0xdead), 0, 1, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::MemsafetyDeref, 1, "main"); +} + +#[test] +fn mode_none_reports_nothing() { + let _guard = fresh(); + assert_eq!(reports_in(AnalysisMode::None), [P::None; 7]); +} + +#[test] +fn mode_target_reports_nothing_from_checks() { + let _guard = fresh(); + assert_eq!(reports_in(AnalysisMode::Target), [P::None; 7]); +} + +#[test] +fn mode_memtrack_reports_memory_errors_only() { + let _guard = fresh(); + assert_eq!( + reports_in(AnalysisMode::Memtrack), + [ + P::MemsafetyDeref, + P::MemsafetyDeref, + P::MemsafetyFree, + P::MemsafetyFree, + P::None, + P::None, + P::MemsafetyMemtrack, + ] + ); +} + +#[test] +fn mode_memcleanup_reports_only_the_leak_as_memcleanup() { + let _guard = fresh(); + let mut expected = [P::None; 7]; + expected[6] = P::MemsafetyMemcleanup; + assert_eq!(reports_in(AnalysisMode::Memcleanup), expected); +} + +#[test] +fn mode_overflow_reports_overflow_only() { + let _guard = fresh(); + let mut expected = [P::None; 7]; + expected[4] = P::Overflow; + assert_eq!(reports_in(AnalysisMode::Overflow), expected); +} + +#[test] +fn mode_assert_reports_assert_only() { + let _guard = fresh(); + let mut expected = [P::None; 7]; + expected[5] = P::Assert; + assert_eq!(reports_in(AnalysisMode::Assert), expected); +} + +#[test] +fn unset_mode_keeps_every_check_active() { + let _guard = fresh(); + let mut out = [P::None; 7]; + // Same calls as `reports_in`, without `map2check_set_mode`. + let runs: [&dyn Fn(); 7] = [ + // SAFETY (all closures): as in `reports_in`. + &|| unsafe { map2check_check_load(addr(0xdead), 1, 0, 4, c"main".as_ptr()) }, + &|| unsafe { map2check_check_deref(addr(0xdead), 0, 1, c"main".as_ptr()) }, + &|| unsafe { map2check_check_free(c"p".as_ptr(), addr(0xdead), 0, 1, c"main".as_ptr()) }, + &|| unsafe { map2check_check_free_resolved_address(addr(0xdead), 1, c"main".as_ptr(), 0) }, + &|| unsafe { map2check_binop_add_int(i32::MAX, 1, 1, 0, c"main".as_ptr()) }, + &|| unsafe { map2check_is_valid_assert(1, c"main".as_ptr(), 0) }, + &|| { + // SAFETY: fake address is never dereferenced. + unsafe { map2check_map_malloc(addr(0x3000), 8) }; + map2check_check_mem_endprog(); + }, + ]; + for (slot, run) in out.iter_mut().zip(runs) { + state::reset(); + map2check_init(); + run(); + *slot = snapshot(|s| s.result.property); + } + assert_eq!(state::config().mode, None); + assert_eq!( + out, + [ + P::MemsafetyDeref, + P::MemsafetyDeref, + P::MemsafetyFree, + P::MemsafetyFree, + P::Overflow, + P::Assert, + P::MemsafetyMemtrack, + ] + ); +} + +#[test] +fn memcleanup_option_is_an_alias_of_memcleanup_mode_for_the_leak() { + let _guard = fresh(); + map2check_set_memcleanup(); + start(AnalysisMode::Memtrack); + // SAFETY: fake address is never dereferenced. + unsafe { map2check_map_malloc(addr(0x3000), 8) }; + map2check_check_mem_endprog(); + assert_violated(P::MemsafetyMemcleanup, 0, "end_of_program"); +} + +#[test] +fn memcleanup_mode_records_release_on_resolved_free() { + // develop: the memcleanup free never fails but still marks the block + // free (AnalysisModeMemcleanup.c:160-179), so it is no leak at exit. + let _guard = fresh(); + start(AnalysisMode::Memcleanup); + // SAFETY: C-string literal is valid; fake address is never dereferenced. + unsafe { + map2check_map_malloc(addr(0x3000), 8); + map2check_check_free_resolved_address(addr(0x3000), 2, c"main".as_ptr(), 0); + } + map2check_check_mem_endprog(); + assert_ok(); +} + +#[test] +fn set_mode_is_configuration_kept_by_init_and_reset() { + let _guard = fresh(); + for mode in MODES { + map2check_set_mode(mode); + map2check_init(); + assert_eq!(state::config().mode, Some(mode)); + map2check_reset(); + assert_eq!(state::config().mode, Some(mode)); + } +} + +#[test] +fn reset_keeps_selected_mode() { + let _guard = fresh(); + map2check_set_mode(AnalysisMode::Overflow); + map2check_init(); + map2check_reset(); + assert_eq!(state::config().mode, Some(AnalysisMode::Overflow)); +} + +#[test] +fn reach_target_sets_target_reached() { + let _guard = fresh(); + map2check_set_mode(AnalysisMode::Target); + map2check_init(); + // SAFETY: C-string literal is valid and null-terminated. + unsafe { mark_target_reached(c"reach_error".as_ptr(), 0, 12) }; + assert_violated(ViolatedProperty::Reachability, 12, "reach_error"); +} + +#[test] +fn map2check_reach_target_records_and_returns() { + // develop: map2check_target_function writes TARGET_REACHED and exits + // (Map2CheckFunctions.c:61-65). Here it records and returns; ending the + // process belongs to Task 2.4. + let _guard = fresh(); + start(AnalysisMode::Target); + // SAFETY: C-string literal is valid and null-terminated. + unsafe { map2check_reach_target(c"reach_error".as_ptr(), 0, 12) }; + assert_violated(ViolatedProperty::Reachability, 12, "reach_error"); + assert_eq!(get_current_property(), ViolatedProperty::Reachability); +} + +#[test] +fn reach_target_keeps_the_first_violation() { + let _guard = fresh(); + start(AnalysisMode::Target); + // SAFETY: C-string literals are valid and null-terminated. + unsafe { + set_false_result(0, 3, c"f".as_ptr()); + map2check_reach_target(c"reach_error".as_ptr(), 0, 12); + } + assert_violated(ViolatedProperty::Overflow, 3, "f"); +} + +#[test] +fn reach_target_works_in_every_mode_and_accepts_null_name() { + // map2check_target_function lives in Map2CheckFunctions.c, linked in + // every mode, so it is not gated by the mode. + let _guard = fresh(); + for mode in MODES { + start(mode); + // SAFETY: null is an accepted name. + unsafe { map2check_reach_target(std::ptr::null(), 0, 7) }; + assert_violated(ViolatedProperty::Reachability, 7, ""); + } +} + +#[test] +fn set_false_result_accepts_the_appended_properties() { + let _guard = fresh(); + // SAFETY: C-string literal is valid and null-terminated. + unsafe { set_false_result(8, 4, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::Assert, 4, "main"); + state::reset(); + // SAFETY: as above. + unsafe { set_false_result(9, 0, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::Unknown, 0, "main"); +} diff --git a/rust/src/state.rs b/rust/src/state.rs index 0eaff75..0811a06 100644 --- a/rust/src/state.rs +++ b/rust/src/state.rs @@ -4,6 +4,7 @@ use std::sync::{LazyLock, Mutex}; use crate::{ + analysismode::mode::AnalysisMode, bbtrack::BasicBlockEntry, caller::{AnalysisResult, ViolatedProperty}, error::Map2CheckError, @@ -18,9 +19,12 @@ static CONFIG: Mutex = Mutex::new(Config::new()); /// Run configuration, kept apart from the analysis state: it changes only /// through its setters, so `map2check_init()` and `map2check_reset()` never -/// clear it. Task 2.3 adds the analysis mode here. +/// clear it. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct Config { + /// Set by `map2check_set_mode()`. `None` means unset: every check is + /// active and the leak tag follows `memcleanup`, as before Task 2.3. + pub mode: Option, /// Set by `map2check_set_memcleanup()`: a leak at exit is reported as /// `MemsafetyMemcleanup` instead of `MemsafetyMemtrack`. pub memcleanup: bool, @@ -31,6 +35,7 @@ pub struct Config { impl Config { pub const fn new() -> Self { Self { + mode: None, memcleanup: false, null_is_valid: false, } @@ -108,20 +113,29 @@ fn ensure_state() -> &'static Mutex { &ANALYSIS_STATE } -/// Starts a fresh analysis: forces the singleton into existence and clears -/// every counter, container, result and flag, so a second `map2check_init()` -/// in the same process never sees data from an earlier analysis. +/// Starts a fresh analysis: forces the singleton into existence and runs the +/// `analysis_init` of the selected mode, which clears every counter, +/// container, result and flag, so a second `map2check_init()` in the same +/// process never sees data from an earlier analysis. The configuration, +/// including the mode, is kept. pub fn init() { - reset(); + let mode = config().mode; + with_state_recovering(|s| crate::analysismode::mode::analysis_init(mode, s)); } pub fn reset() { + with_state_recovering(AnalysisState::reset); +} + +/// Applies `f` to the state, recovering the lock if an earlier panic +/// poisoned it. +fn with_state_recovering(f: impl FnOnce(&mut AnalysisState)) { let state = ensure_state(); match state.lock() { - Ok(mut guard) => guard.reset(), + Ok(mut guard) => f(&mut guard), Err(poisoned) => { let mut guard = poisoned.into_inner(); - guard.reset(); + f(&mut guard); state.clear_poison(); } } diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index 3d0553a..d5b4710 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -55,6 +55,9 @@ static void run_pre_init_boundary(void) { vcc_reset_meta_data(); map2check_set_memcleanup(); map2check_set_null_is_valid(); + /* map2check_set_mode is configuration that no API unsets, so it is + * exercised last, in test_analysis_modes(). */ + map2check_reach_target("main", 0, 1); map2check_map_alloca("", &stack_obj, 0, 0, 1, 0); map2check_map_non_static_alloca("", &stack_obj, 0, 0, 1, 0); @@ -151,6 +154,7 @@ static void run_bad_inputs(void) { map2check_check_free(NULL, NULL, 0, 3, NULL); map2check_check_free_resolved_address(NULL, 3, NULL, 1); map2check_check_mem_endprog(); + map2check_reach_target(NULL, -1, -1); /* memtrack with null/empty strings */ map2check_map_alloca(NULL, NULL, 4, 4, 4, 1); @@ -348,6 +352,40 @@ static int test_result_and_control_flags(void) { return 0; } +/* Task 2.3: the mode selected at run time gates the checks. Runs last + * because no API returns the mode to unset. */ +static int test_analysis_modes(void) { + map2check_set_mode(AnalysisMode_None); + map2check_init(); + map2check_check_deref((void *)0xdead, 0, 1, "main"); + if (get_current_property() != None) { + return 1; + } + + map2check_set_mode(AnalysisMode_Memtrack); + map2check_init(); + map2check_check_deref((void *)0xdead, 0, 1, "main"); + if (get_current_property() != MemsafetyDeref) { + return 2; + } + + map2check_set_mode(AnalysisMode_Assert); + map2check_init(); + map2check_binop_add_int(2147483647, 1, 2, 0, "main"); + if (get_current_property() != None) { + return 3; + } + + map2check_set_mode(AnalysisMode_Target); + map2check_init(); + map2check_reach_target("reach_error", 0, 12); + if (get_current_property() != Reachability) { + return 4; + } + map2check_reset(); + return 0; +} + int main(void) { if (MAP2CHECK_VERSION[0] == '\0') { return 1; @@ -384,6 +422,10 @@ int main(void) { } } + if (test_analysis_modes() != 0) { + return 1; + } + map2check_success(); return 0; } From 854c1e8a3d8d42387b34756a8e6b4fc18b97dd6f Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 21:44:38 -0400 Subject: [PATCH 21/63] fix: validate the analysis mode and match develop's memcleanup verdict map2check_set_mode now takes an int32_t. A value outside 0-5 is ignored with a diagnostic instead of being undefined behaviour. The header still declares enum AnalysisMode for the values. Memcleanup mode now reports a leak only when the last leaked block is still referenced by a pointer variable, as develop does. The allocation and list logs are replayed from the memtrack entries. Memtrack mode already matched develop. Remove the unused MemCleanupChecker. --- cbindgen.toml | 4 +- docs/api-deltas.md | 10 +- docs/cutover/abi-mapping.tsv | 2 +- docs/semantic-deltas.md | 73 +++++---- rust/src/analysismode/memory.rs | 53 ------- rust/src/analysismode/mode.rs | 29 +++- rust/src/ffi.rs | 32 +++- rust/src/ffi_tests.rs | 197 ++++++++++++++++++++++-- rust/src/memtrack/mod.rs | 255 ++++++++++++++++++++++++++++++++ tests/ffi_harness.c | 26 ++++ 10 files changed, 572 insertions(+), 109 deletions(-) diff --git a/cbindgen.toml b/cbindgen.toml index 868f34e..be76d31 100644 --- a/cbindgen.toml +++ b/cbindgen.toml @@ -8,7 +8,9 @@ documentation_style = "doxy" [export] prefix = "" -include = [] +# `map2check_set_mode` takes an `int32_t`; export the enum so the header +# still names the mode values. +include = ["AnalysisMode"] exclude = [] [parse] diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 41a37ab..1fd558e 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -54,7 +54,7 @@ build with `--features libfuzzer`. | `vcc_reset_meta_data` | `void vcc_reset_meta_data(void)` | `vcc_reset_meta_data` | same | Host/test-only: the Map2Check passes never call it. It clears a recorded violation by design, bypassing the first-violation rule (SD-03). | | `map2check_set_memcleanup` | `void map2check_set_memcleanup(void)` | `map2check_set_memcleanup` | same | Selects `MEMSAFETY_MEMCLEANUP` instead of `MEMSAFETY_MEMTRACK` for the end-of-program leak (SD-04): an alias of the memcleanup mode for the leak tag, in every mode that checks leaks (SD-14). Kept by `map2check_init()` and `map2check_reset()`. | | `map2check_set_null_is_valid` | `void map2check_set_null_is_valid(void)` | `map2check_set_null_is_valid` | same | Makes a NULL load safe (SD-05). Kept by `map2check_init()` and `map2check_reset()`. | -| `map2check_set_mode` | `void map2check_set_mode(enum AnalysisMode mode)` | — | new | Selects the analysis mode at run time (`AnalysisMode_None` 0, `_Memtrack` 1, `_Memcleanup` 2, `_Overflow` 3, `_Assert` 4, `_Target` 5); develop selects it at link time. Called before `map2check_init()`; kept by `map2check_init()` and `map2check_reset()`. Never called: mode unset, every check active. See “Analysis modes” (SD-14). A value outside 0–5 is undefined behaviour. | +| `map2check_set_mode` | `void map2check_set_mode(int32_t mode)` | — | new | Selects the analysis mode at run time; `mode` is one of the `enum AnalysisMode` values the header declares (`AnalysisMode_None` 0, `_Memtrack` 1, `_Memcleanup` 2, `_Overflow` 3, `_Assert` 4, `_Target` 5). Develop selects it at link time. Called before `map2check_init()`; kept by `map2check_init()` and `map2check_reset()`. Never called: mode unset, every check active. Any other value is ignored with a diagnostic on stderr and leaves the mode unchanged. See “Analysis modes” (SD-14). | | `map2check_reach_target` | `void map2check_reach_target(const char *func, int32_t scope, int32_t line)` | — | new | Replaces the pass symbol `map2check_target_function`. Records `REACHABILITY` (develop `TARGET-REACHED`) at `line` in `func`, in every mode; `scope` is ignored. Does not end the process (SD-15). | | `map2check_save_nondet_log_int` | `non_det_log_t *map2check_save_nondet_log_int(int line, int scope, enum var_type_t type_var, int *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_uint` | `non_det_log_t *map2check_save_nondet_log_uint(int line, int scope, enum var_type_t type_var, unsigned int *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | @@ -88,7 +88,7 @@ build with `--features libfuzzer`. | `map2check_check_free` | `void map2check_check_free(const char *_name, const void *ptr, uint32_t _scope, uint32_t line, const char *function_name)` | `map2check_check_free` | same | `_name` and `_scope` are ignored. Untracked address is a violation (SD-06). Active in unset and memtrack modes only (SD-14). | | `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; records the release when the check passes, unlike `map2check_check_free`; untracked address is a violation (SD-06); see also SD-13. The check is active in unset and memtrack modes only; when inactive the release is still recorded (SD-14). | | `map2check_check_deref` | `void map2check_check_deref(const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_deref` | same | Name collides with the pass ABI `map2check_check_deref(i64, ptr)`; see the pass matrix (#21). Active in unset and memtrack modes only (SD-14). | -| `map2check_check_mem_endprog` | `void map2check_check_mem_endprog(void)` | `map2check_check_mem_endprog` | changed | Leak algorithm rewritten. Runs `analysis_is_program_correct` of the mode: unset and memtrack report memtrack or memcleanup by option, memcleanup mode reports memcleanup, the other modes nothing (SD-04, SD-14). | +| `map2check_check_mem_endprog` | `void map2check_check_mem_endprog(void)` | `map2check_check_mem_endprog` | changed | Leak algorithm rewritten. Runs `analysis_is_program_correct` of the mode: unset and memtrack report memtrack or memcleanup by option, memcleanup mode reports memcleanup only for a still-referenced last leak, the other modes nothing (SD-04, SD-14). | | `LLVMFuzzerTestOneInput` | `int32_t LLVMFuzzerTestOneInput(const uint8_t *data, uintptr_t size)` | `LLVMFuzzerTestOneInput` | same | Feature `libfuzzer`. Resets the analysis state on every iteration (SD-11). | | `__VERIFIER_nondet_bool` | `bool __VERIFIER_nondet_bool(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | | `__VERIFIER_nondet_char` | `int8_t __VERIFIER_nondet_char(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | @@ -169,12 +169,14 @@ per-mode `match` is in `rust/src/analysismode/mode.rs`. Tracking calls |---|---|---|---| | unset (`map2check_set_mode` never called) | — | all | leak → `MEMSAFETY_MEMTRACK`, or `MEMSAFETY_MEMCLEANUP` with the option | | `AnalysisMode_None` (0) | `COVER_BRANCHES_MODE` | none | nothing | -| `AnalysisMode_Memtrack` (1) | `MEMTRACK_MODE` | load, deref, free, free resolved | leak → `MEMSAFETY_MEMTRACK`, or `MEMSAFETY_MEMCLEANUP` with the option | -| `AnalysisMode_Memcleanup` (2) | `MEMCLEANUP_MODE` | none (free resolved still records the release) | leak → `MEMSAFETY_MEMCLEANUP` | +| `AnalysisMode_Memtrack` (1) | `MEMTRACK_MODE` | load, deref, free, free resolved | any live heap block (reachable or not) → `MEMSAFETY_MEMTRACK`, or `MEMSAFETY_MEMCLEANUP` with the option | +| `AnalysisMode_Memcleanup` (2) | `MEMCLEANUP_MODE` | none (free resolved still records the release) | `MEMSAFETY_MEMCLEANUP` only when the last leaked block is still referenced by a pointer variable, as in develop; an unreachable leak gives no verdict | | `AnalysisMode_Overflow` (3) | `OVERFLOW_MODE` | `map2check_binop_*` | nothing | | `AnalysisMode_Assert` (4) | `ASSERT_MODE` | `map2check_is_valid_assert` (`ASSERT`) | nothing | | `AnalysisMode_Target` (5) | `REACHABILITY_MODE` | none | nothing | +The memcleanup option only changes the tag of the unset/memtrack leak +verdict; it does not apply the memcleanup-mode reachability rule. `map2check_reach_target` and `set_false_result` record in every mode. `ViolatedProperty` gained `Assert` (8) and `Unknown` (9), appended after `None`; `Reachability` is the target-reached property. diff --git a/docs/cutover/abi-mapping.tsv b/docs/cutover/abi-mapping.tsv index 2e1aec3..7f89cef 100644 --- a/docs/cutover/abi-mapping.tsv +++ b/docs/cutover/abi-mapping.tsv @@ -1,5 +1,5 @@ c_symbol core_symbol owner_task notes -map2check_init map2check_set_mode+map2check_init 2.3 C: (i32 isSvComp) → núcleo: set_mode(AnalysisMode)+init(void) (Task 2.3; o modo vinha do .bc ligado, caller.cpp:575-612); o arg int isSvComp é morto: Map2CheckFunctions.c:35 grava sv_comp e nenhum arquivo o lê +map2check_init map2check_set_mode+map2check_init 2.3 C: (i32 isSvComp) → núcleo: set_mode(int32_t, valores de enum AnalysisMode 0-5; outro valor é ignorado)+init(void) (Task 2.3; o modo vinha do .bc ligado, caller.cpp:575-612); o arg int isSvComp é morto: Map2CheckFunctions.c:35 grava sv_comp e nenhum arquivo o lê map2check_success map2check_success - C: () → núcleo: () (LibraryFunctions.hpp:48) map2check_track_bb map2check_save_basic_block_log - C: (i32 line,ptr fn) → núcleo: (line:i32,fn); mesma ordem (LibraryFunctions.hpp:54, ffi.rs:111); retorno do núcleo é bbtrack_log_t*, ignorado map2check_target_function map2check_reach_target 2.3 C: (ptr fn,i32 scope,i32 line) (TargetPass.cpp:89); C runtime grava TARGET_REACHED e aborta (Map2CheckFunctions.c:61-65); núcleo: map2check_reach_target(const char*,i32,i32) (Task 2.3) registra Reachability (=TARGET-REACHED) em qualquer modo, ignora scope e retorna; C encerra na primeira violação; núcleo registra e continua (SD-03/#36, SD-15); término do processo é a Task 2.4 diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 7354471..06abef3 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -393,40 +393,61 @@ map2check_check_free_resolved_address((void *)0x1000, 3, "main", 0); /* None; d ### SD-14 — The analysis mode is chosen at run time; unset keeps every check **Classification:** intentional -**Tests:** `rust/src/ffi_tests.rs::unset_mode_keeps_every_check_active`, `rust/src/ffi_tests.rs::mode_none_reports_nothing`, `rust/src/ffi_tests.rs::mode_memtrack_reports_memory_errors_only`, `rust/src/ffi_tests.rs::mode_memcleanup_reports_only_the_leak_as_memcleanup`, `rust/src/ffi_tests.rs::mode_overflow_reports_overflow_only`, `rust/src/ffi_tests.rs::mode_assert_reports_assert_only`, `rust/src/ffi_tests.rs::mode_target_reports_nothing_from_checks`, `rust/src/ffi_tests.rs::memcleanup_option_is_an_alias_of_memcleanup_mode_for_the_leak`, `rust/src/ffi_tests.rs::is_valid_assert_false_reports_assert`, `rust/src/analysismode/mode.rs::program_correct_memcleanup_reports_leak_as_memcleanup` +**Tests:** `rust/src/ffi_tests.rs::unset_mode_keeps_every_check_active`, `rust/src/ffi_tests.rs::mode_none_reports_nothing`, `rust/src/ffi_tests.rs::mode_memtrack_reports_memory_errors_only`, `rust/src/ffi_tests.rs::mode_memcleanup_reports_only_the_leak_as_memcleanup`, `rust/src/ffi_tests.rs::mode_overflow_reports_overflow_only`, `rust/src/ffi_tests.rs::mode_assert_reports_assert_only`, `rust/src/ffi_tests.rs::mode_target_reports_nothing_from_checks`, `rust/src/ffi_tests.rs::memcleanup_option_is_an_alias_of_memcleanup_mode_for_the_leak`, `rust/src/ffi_tests.rs::is_valid_assert_false_reports_assert`, `rust/src/ffi_tests.rs::set_mode_ignores_out_of_range_values`, `rust/src/ffi_tests.rs::memcleanup_mode_leak_never_stored_has_no_verdict`, `rust/src/ffi_tests.rs::memcleanup_mode_checks_only_the_last_leak`, `rust/src/ffi_tests.rs::memtrack_mode_reports_any_leak_reachable_or_not` **Rust:** `map2check_set_mode()` selects the mode before `map2check_init()`; -init and reset keep it. Each mode activates the checks that its -`AnalysisMode*.c` defines in develop; a check of another mode records -nothing. Tracking calls record in every mode, and an inactive -`map2check_check_free_resolved_address` still records the release. When -`map2check_set_mode()` is never called the mode is unset: every check is -active and the leak tag follows the memcleanup option, as before Task 2.3. -The memcleanup option is an alias of the memcleanup mode for the leak tag, -so memtrack mode plus the option reports `MEMSAFETY_MEMCLEANUP`. A failed -assert reports `ASSERT` instead of `REACHABILITY`. In memcleanup mode a leak -is any live allocation by the Rust leak rule; develop also requires -`is_memcleanup_error` (`ListLog.c:40-65`) to hold. - -**C runtime of Map2Check `develop`:** the mode is fixed at link time -(`modules/frontend/caller.cpp:575-612`): one of `AnalysisModeNone.c` -(cover-branches and reachability), `AnalysisModeMemtrack.c`, -`AnalysisModeMemcleanup.c`, `AnalysisModeOverflow.c` or -`AnalysisModeAssert.c` is linked, so calls of another mode are never emitted -and there is no unset mode. `map2check_assert` writes `ASSERT` -(`AnalysisModeAssert.c:18-23`, `PropertyGenerator.c:92-97`). The memcleanup -mode never fails a load, deref or free (`AnalysisModeMemcleanup.c:158`, -`:293`, `:166`, `:343`). +init and reset keep it, and a value outside 0–5 is ignored with a +diagnostic. With a mode set, the checks and the end-of-program verdict +follow the `AnalysisMode*.c` of that mode, at parity with develop: + +- a check of another mode records nothing; +- memtrack reports any live heap block at exit; +- memcleanup reports `MEMSAFETY_MEMCLEANUP` only when the last leaked + block, in allocation order, is still referenced by a pointer variable's + latest stored value. + +The differences that remain are these: + +- **Unset mode.** When `map2check_set_mode()` is never called the mode is + unset: every check is active and the leak tag follows the memcleanup + option, as before Task 2.3. Develop has no unset mode. +- **Memcleanup option.** The option only retags the unset/memtrack leak + verdict as `MEMSAFETY_MEMCLEANUP`. It does not apply the memcleanup-mode + reachability rule. +- **Tracking calls** record in every mode. +- **Free records.** The Rust free record has no pointer-variable address. + So the memcleanup replay does not rebuild the FREE row that develop's + `map2check_free` appends for the pointer variable + (`AnalysisModeMemcleanup.c:336-341`). That row matters only when a freed + address is allocated again. + +**C runtime of Map2Check `develop`:** + +- **Mode selection.** The mode is fixed at link time + (`modules/frontend/caller.cpp:575-612`), so calls of another mode are + never emitted. +- **Memtrack verdict.** Memtrack reports FALSE_MEMTRACK when + `valid_allocation_log` (`AllocationLog.c:69-101`) returns non-zero + (`AnalysisModeMemtrack.c:138-146`). +- **Memcleanup verdict.** Memcleanup additionally requires + `is_memcleanup_error` (`ListLog.c:40-65`) for that last address + (`AnalysisModeMemcleanup.c:138-148`), over the list log that + `map2check_add_store_pointer` and `update_reference_list_log` maintain + (`:181-291`). +- **Asserts.** `map2check_assert` writes `ASSERT` + (`AnalysisModeAssert.c:18-23`). **Reproduce:** ```c -map2check_set_mode(AnalysisMode_None); +map2check_set_mode(AnalysisMode_Memcleanup); map2check_init(); -map2check_check_deref((void *)0xdead, 0, 1, "main"); /* None */ -map2check_set_mode(AnalysisMode_Memtrack); +map2check_map_malloc((void *)0x3000, 8); +map2check_check_mem_endprog(); /* None: nothing points to it */ map2check_init(); -map2check_check_deref((void *)0xdead, 0, 1, "main"); /* MemsafetyDeref */ +map2check_map_malloc((void *)0x3000, 8); +map2check_map_store_pointer((void *)0x10, (void *)0x3000, 0, "p", 1, "main"); +map2check_check_mem_endprog(); /* MemsafetyMemcleanup */ ``` ### SD-15 — Reaching the target records the violation and returns diff --git a/rust/src/analysismode/memory.rs b/rust/src/analysismode/memory.rs index 41c9d4e..1f31237 100644 --- a/rust/src/analysismode/memory.rs +++ b/rust/src/analysismode/memory.rs @@ -122,33 +122,6 @@ impl VccChecker for DerefChecker { } } -// ── End-of-program memory cleanup check ────────────────────────────────────── - -/// Checks for memory leaks at end of program. -/// Replaces `map2check_check_mem_endprog`. -pub struct MemCleanupChecker { - /// `true` reports `MemsafetyMemcleanup`, `false` `MemsafetyMemtrack`. - pub memcleanup: bool, -} - -impl VccChecker for MemCleanupChecker { - fn check(&self, _ctx: &VccContext) -> Result { - state::with_state(|s| { - if memtrack::has_memcleanup_error(&s.memtrack) { - Ok(VccOutcome::Violated { - property: if self.memcleanup { - ViolatedProperty::MemsafetyMemcleanup - } else { - ViolatedProperty::MemsafetyMemtrack - }, - }) - } else { - Ok(VccOutcome::Safe) - } - }) - } -} - #[cfg(test)] mod tests { use super::*; @@ -339,30 +312,4 @@ mod tests { violated(ViolatedProperty::MemsafetyDeref) ); } - - #[test] - fn mem_cleanup_flags_leaked_allocation() { - let _g = with_entries(vec![stack(0x30), heap(0x10, 0x1000, 16, false)]); - assert_eq!( - MemCleanupChecker { memcleanup: true } - .check(&ctx()) - .unwrap(), - violated(ViolatedProperty::MemsafetyMemcleanup) - ); - } - - #[test] - fn mem_cleanup_accepts_freed_allocation_and_stack_vars() { - let _g = with_entries(vec![ - stack(0x30), - heap(0x10, 0x1000, 16, false), - heap(0x10, 0x1000, 16, true), - ]); - assert_eq!( - MemCleanupChecker { memcleanup: true } - .check(&ctx()) - .unwrap(), - VccOutcome::Safe - ); - } } diff --git a/rust/src/analysismode/mode.rs b/rust/src/analysismode/mode.rs index 6b737cd..8f22407 100644 --- a/rust/src/analysismode/mode.rs +++ b/rust/src/analysismode/mode.rs @@ -136,12 +136,16 @@ pub fn analysis_is_program_correct( match config.mode { // Unset: the pre-Task-2.3 leak check, tagged by the option (SD-04). None => leak(memtrack_tag), - // AnalysisModeMemtrack.c:138-146 — a live allocation is FALSE_MEMTRACK. + // AnalysisModeMemtrack.c:138-146 — FALSE_MEMTRACK when + // valid_allocation_log returns non-zero, i.e. any heap block is still + // live; no reachability rule. `has_memcleanup_error` is that test. Some(AnalysisMode::Memtrack) => leak(memtrack_tag), - // AnalysisModeMemcleanup.c:138-148 — a live allocation is - // FALSE_MEMCLEANUP (develop also asks `is_memcleanup_error`, - // ListLog.c:40-65; Rust uses its own leak rule, see SD-14). - Some(AnalysisMode::Memcleanup) => leak(ViolatedProperty::MemsafetyMemcleanup), + // AnalysisModeMemcleanup.c:138-148 — FALSE_MEMCLEANUP only when the + // last leaked address (valid_allocation_log, AllocationLog.c:69-101) + // is still referenced by a pointer variable (is_memcleanup_error, + // ListLog.c:40-65). An unreachable leak gives no verdict. + Some(AnalysisMode::Memcleanup) => memtrack::has_develop_memcleanup_error(&state.memtrack) + .then_some(ViolatedProperty::MemsafetyMemcleanup), // AnalysisModeNone.c:19 — TRUE. Some(AnalysisMode::None) => None, // AnalysisModeOverflow.c:283 — TRUE. @@ -413,7 +417,20 @@ mod tests { #[test] fn program_correct_memcleanup_reports_leak_as_memcleanup() { - let s = leaked_state(); + // Develop reports only a leak still referenced by a pointer + // (AnalysisModeMemcleanup.c:138-148, ListLog.c:40-65). + assert_eq!( + analysis_is_program_correct( + config(Some(AnalysisMode::Memcleanup), false), + &leaked_state() + ), + None, + "an unreferenced leak is no memcleanup error" + ); + let mut s = leaked_state(); + s.memtrack.push(MemTrackEntry::new( + 2, 1, 0, 0x10, 0x1000, false, false, "p", "main", 0, 1, false, + )); let c = config(Some(AnalysisMode::Memcleanup), false); assert_eq!( analysis_is_program_correct(c, &s), diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index c4d6209..90842de 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -272,23 +272,39 @@ pub extern "C" fn map2check_set_null_is_valid() { /// Selects the analysis mode, as the frontend of Map2Check `develop` does by /// linking one `AnalysisMode*.c` per run (`caller.cpp:575-612`). The -/// instrumented program calls it before `map2check_init()`. The mode is -/// configuration: `map2check_init()` and `map2check_reset()` keep it. Until -/// it is called the mode is unset and every check is active. Per-mode effects -/// are in `rust/src/analysismode/mode.rs`. -/// -/// `mode` must be one of the `AnalysisMode` values (0 to 5); any other value -/// is undefined behaviour, as for any C enum passed to Rust. +/// instrumented program calls it before `map2check_init()` with one of the +/// `AnalysisMode` values (`AnalysisMode_None` 0 to `AnalysisMode_Target` 5). +/// Any other value is ignored with a diagnostic on stderr and leaves the mode +/// as it was. The mode is configuration: `map2check_init()` and +/// `map2check_reset()` keep it. Until a valid value is set the mode is unset +/// and every check is active. Per-mode effects are in +/// `rust/src/analysismode/mode.rs`. #[no_mangle] -pub extern "C" fn map2check_set_mode(mode: AnalysisMode) { +pub extern "C" fn map2check_set_mode(mode: i32) { ffi_guard( || { + let Some(mode) = analysis_mode_from_raw(mode) else { + eprintln!("[map2check] invalid analysis mode: {mode}"); + return; + }; state::update_config(|c| c.mode = Some(mode)); }, (), ); } +fn analysis_mode_from_raw(raw: i32) -> Option { + match raw { + 0 => Some(AnalysisMode::None), + 1 => Some(AnalysisMode::Memtrack), + 2 => Some(AnalysisMode::Memcleanup), + 3 => Some(AnalysisMode::Overflow), + 4 => Some(AnalysisMode::Assert), + 5 => Some(AnalysisMode::Target), + _ => None, + } +} + /// Records that the target function was reached, without ending the process. /// Same property as `map2check_target_function` of the `develop` C runtime /// (`TARGET_REACHED`, `Map2CheckFunctions.c:61-65`), which then ends the run; diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 7d1f412..f53e9ed 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -1260,7 +1260,7 @@ const MODES: [AnalysisMode; 6] = [ /// Starts an analysis in `mode`, as the instrumented program will. fn start(mode: AnalysisMode) { state::reset(); - map2check_set_mode(mode); + map2check_set_mode(mode as i32); map2check_init(); } @@ -1278,8 +1278,19 @@ fn reports_in(mode: AnalysisMode) -> [ViolatedProperty; 7] { &|| unsafe { map2check_binop_add_int(i32::MAX, 1, 1, 0, c"main".as_ptr()) }, &|| unsafe { map2check_is_valid_assert(1, c"main".as_ptr(), 0) }, &|| { - // SAFETY: fake address is never dereferenced. - unsafe { map2check_map_malloc(addr(0x3000), 8) }; + // A leak still referenced by `p`, so it counts in every leak mode. + // SAFETY: fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x3000), 8); + map2check_map_store_pointer( + addr(0x10), + addr(0x3000), + 0, + c"p".as_ptr(), + 1, + c"main".as_ptr(), + ); + } map2check_check_mem_endprog(); }, ]; @@ -1296,7 +1307,7 @@ use ViolatedProperty as P; #[test] fn mode_none_never_reports_memory_errors() { let _guard = fresh(); - map2check_set_mode(AnalysisMode::None); + map2check_set_mode(AnalysisMode::None as i32); map2check_init(); // SAFETY: C-string literal is valid; fake address is never dereferenced. unsafe { map2check_check_deref(addr(0xdead), 0, 1, c"main".as_ptr()) }; @@ -1306,7 +1317,7 @@ fn mode_none_never_reports_memory_errors() { #[test] fn mode_memtrack_reports_invalid_deref() { let _guard = fresh(); - map2check_set_mode(AnalysisMode::Memtrack); + map2check_set_mode(AnalysisMode::Memtrack as i32); map2check_init(); // SAFETY: C-string literal is valid; fake address is never dereferenced. unsafe { map2check_check_deref(addr(0xdead), 0, 1, c"main".as_ptr()) }; @@ -1380,8 +1391,19 @@ fn unset_mode_keeps_every_check_active() { &|| unsafe { map2check_binop_add_int(i32::MAX, 1, 1, 0, c"main".as_ptr()) }, &|| unsafe { map2check_is_valid_assert(1, c"main".as_ptr(), 0) }, &|| { - // SAFETY: fake address is never dereferenced. - unsafe { map2check_map_malloc(addr(0x3000), 8) }; + // A leak still referenced by `p`, so it counts in every leak mode. + // SAFETY: fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x3000), 8); + map2check_map_store_pointer( + addr(0x10), + addr(0x3000), + 0, + c"p".as_ptr(), + 1, + c"main".as_ptr(), + ); + } map2check_check_mem_endprog(); }, ]; @@ -1436,7 +1458,7 @@ fn memcleanup_mode_records_release_on_resolved_free() { fn set_mode_is_configuration_kept_by_init_and_reset() { let _guard = fresh(); for mode in MODES { - map2check_set_mode(mode); + map2check_set_mode(mode as i32); map2check_init(); assert_eq!(state::config().mode, Some(mode)); map2check_reset(); @@ -1447,7 +1469,7 @@ fn set_mode_is_configuration_kept_by_init_and_reset() { #[test] fn reset_keeps_selected_mode() { let _guard = fresh(); - map2check_set_mode(AnalysisMode::Overflow); + map2check_set_mode(AnalysisMode::Overflow as i32); map2check_init(); map2check_reset(); assert_eq!(state::config().mode, Some(AnalysisMode::Overflow)); @@ -1456,7 +1478,7 @@ fn reset_keeps_selected_mode() { #[test] fn reach_target_sets_target_reached() { let _guard = fresh(); - map2check_set_mode(AnalysisMode::Target); + map2check_set_mode(AnalysisMode::Target as i32); map2check_init(); // SAFETY: C-string literal is valid and null-terminated. unsafe { mark_target_reached(c"reach_error".as_ptr(), 0, 12) }; @@ -1512,3 +1534,158 @@ fn set_false_result_accepts_the_appended_properties() { unsafe { set_false_result(9, 0, c"main".as_ptr()) }; assert_violated(ViolatedProperty::Unknown, 0, "main"); } + +// ── I-1: map2check_set_mode validates its raw value ──────────────────────── + +#[test] +fn set_mode_accepts_a_valid_raw_value() { + let _guard = fresh(); + map2check_set_mode(2); + assert_eq!(state::config().mode, Some(AnalysisMode::Memcleanup)); +} + +#[test] +fn set_mode_ignores_out_of_range_values() { + let _guard = fresh(); + map2check_set_mode(AnalysisMode::Overflow as i32); + for raw in [-1, 6, i32::MAX] { + map2check_set_mode(raw); + assert_eq!( + state::config().mode, + Some(AnalysisMode::Overflow), + "raw {raw}" + ); + } +} + +#[test] +fn set_mode_out_of_range_before_any_mode_keeps_it_unset() { + let _guard = fresh(); + map2check_set_mode(-1); + assert_eq!(state::config().mode, None); +} + +// ── I-2: memcleanup mode follows develop's last-leak + reference rule ────── + +/// Stores `value` into the pointer variable at `var` (`var = value;`). +fn store(var: usize, value: usize) { + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_map_store_pointer( + addr(var), + addr(value), + 0, + c"p".as_ptr(), + 1, + c"main".as_ptr(), + ) + }; +} + +fn malloc_at(a: usize) { + // SAFETY: fake address is never dereferenced. + unsafe { map2check_map_malloc(addr(a), 8) }; +} + +fn memcleanup_endprog() -> ViolatedProperty { + map2check_check_mem_endprog(); + snapshot(|s| s.result.property) +} + +#[test] +fn memcleanup_mode_leak_still_pointed_to_is_memcleanup() { + let _guard = fresh(); + start(AnalysisMode::Memcleanup); + malloc_at(0x3000); + store(0x10, 0x3000); + map2check_check_mem_endprog(); + assert_violated(P::MemsafetyMemcleanup, 0, "end_of_program"); +} + +#[test] +fn memcleanup_mode_leak_whose_pointer_was_overwritten_has_no_verdict() { + let _guard = fresh(); + start(AnalysisMode::Memcleanup); + malloc_at(0x3000); + store(0x10, 0x3000); + store(0x10, 0); + assert_eq!(memcleanup_endprog(), P::None); +} + +#[test] +fn memcleanup_mode_leak_never_stored_has_no_verdict() { + let _guard = fresh(); + start(AnalysisMode::Memcleanup); + malloc_at(0x3000); + assert_eq!(memcleanup_endprog(), P::None); +} + +#[test] +fn memcleanup_mode_checks_only_the_last_leak() { + // valid_allocation_log returns the last leaked address only + // (AllocationLog.c:69-101): the referenced earlier leak is not checked. + let _guard = fresh(); + start(AnalysisMode::Memcleanup); + malloc_at(0x3000); + malloc_at(0x4000); + store(0x10, 0x3000); + assert_eq!(memcleanup_endprog(), P::None); + + start(AnalysisMode::Memcleanup); + malloc_at(0x3000); + malloc_at(0x4000); + store(0x10, 0x4000); + assert_eq!(memcleanup_endprog(), P::MemsafetyMemcleanup); +} + +#[test] +fn memcleanup_mode_another_pointer_keeps_the_leak_reachable() { + let _guard = fresh(); + start(AnalysisMode::Memcleanup); + malloc_at(0x3000); + store(0x10, 0x3000); + store(0x20, 0x3000); + store(0x10, 0); + assert_eq!(memcleanup_endprog(), P::MemsafetyMemcleanup); +} + +#[test] +fn memcleanup_mode_freed_block_is_no_leak() { + let _guard = fresh(); + start(AnalysisMode::Memcleanup); + malloc_at(0x3000); + store(0x10, 0x3000); + // SAFETY: C-string literals are valid; fake address is never dereferenced. + unsafe { map2check_map_free(c"p".as_ptr(), addr(0x3000), 0, 2, c"main".as_ptr()) }; + assert_eq!(memcleanup_endprog(), P::None); +} + +#[test] +fn memcleanup_mode_free_of_an_old_target_re_points_the_variable_as_develop() { + // update_reference_list_log (AnalysisModeMemcleanup.c:181-240) appends a + // FREE copy of `p = A` when A is freed, so `p` reads as pointing to A + // again and B (the last leak) is no longer referenced by it. + let _guard = fresh(); + start(AnalysisMode::Memcleanup); + malloc_at(0x3000); + malloc_at(0x4000); + store(0x10, 0x3000); + store(0x10, 0x4000); + // SAFETY: C-string literal is valid; fake address is never dereferenced. + unsafe { map2check_check_free_resolved_address(addr(0x3000), 3, c"main".as_ptr(), 0) }; + assert_eq!(memcleanup_endprog(), P::None); +} + +#[test] +fn memtrack_mode_reports_any_leak_reachable_or_not() { + // AnalysisModeMemtrack.c:138-146: any address from valid_allocation_log. + let _guard = fresh(); + start(AnalysisMode::Memtrack); + malloc_at(0x3000); + assert_eq!(memcleanup_endprog(), P::MemsafetyMemtrack); + start(AnalysisMode::Memtrack); + malloc_at(0x3000); + store(0x10, 0x3000); + store(0x10, 0); + assert_eq!(memcleanup_endprog(), P::MemsafetyMemtrack); +} diff --git a/rust/src/memtrack/mod.rs b/rust/src/memtrack/mod.rs index 8634c18..7f4d313 100644 --- a/rust/src/memtrack/mod.rs +++ b/rust/src/memtrack/mod.rs @@ -152,6 +152,186 @@ pub fn has_memcleanup_error(entries: &[MemTrackEntry]) -> bool { }) } +// ── develop's memcleanup verdict ───────────────────────────────────────────── +// +// Memcleanup mode of the Map2Check `develop` C runtime decides the +// end-of-program verdict on two logs that this crate does not keep: the +// allocation log (one row per heap address, updated in place) and the list +// log (pointer-variable rows). `DevelopLogs::replay` rebuilds both from the +// memtrack entries, in order, with the same rules as +// `AnalysisModeMemcleanup.c`, so the verdict can be computed exactly. + +/// `enum MemoryAddressStatus` (`Map2CheckTypes.h:34`). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum AddressStatus { + Static, + Free, + Dynamic, +} + +/// A list-log row: `memory_address`, `memory_address_points_to` and the +/// status that `get_type_from_list_log_row` reads (`ListLog.c:159-168`). +#[derive(Debug, Clone, Copy)] +struct ListRow { + var: usize, + points_to: usize, + status: AddressStatus, +} + +#[derive(Debug, Default)] +struct DevelopLogs { + /// `(addr, is_free)`, one row per address in first-allocation order. + allocation: Vec<(usize, bool)>, + list: Vec, +} + +impl DevelopLogs { + fn replay(entries: &[MemTrackEntry]) -> Self { + let mut logs = Self::default(); + for e in entries { + if e.is_dynamic && !e.is_free { + logs.malloc(e.points_to); + } else if e.is_free { + logs.free(e.points_to); + } else if e.size_destiny == 0 { + logs.store_pointer(e.var_addr, e.points_to); + } + // Stack slots and function addresses go to develop's heap log, + // which the verdict does not read. + } + logs + } + + /// `map2check_malloc` (`AnalysisModeMemcleanup.c:295-313`): updates the + /// row of the address in place or appends one. A NULL result ends the + /// develop run with UNKNOWN (`:297-300`), so it adds no row here. + fn malloc(&mut self, addr: usize) { + if addr == 0 { + return; + } + match self.allocation.iter_mut().rev().find(|(a, _)| *a == addr) { + Some(row) => row.1 = false, + None => self.allocation.push((addr, false)), + } + } + + /// The free paths, `map2check_free` (`:329-356`) and + /// `map2check_free_resolved_address` (`:160-179`): mark the row free and + /// run `update_reference_list_log(addr, FREE)`. `map2check_free` also + /// appends a FREE row for the pointer variable (`:336-341`); the Rust free + /// record carries only the heap address, so that row is not rebuilt. + fn free(&mut self, addr: usize) { + if addr == 0 { + return; + } + if let Some(row) = self.allocation.iter_mut().rev().find(|(a, _)| *a == addr) { + row.1 = true; + } + self.update_reference(addr, AddressStatus::Free); + } + + /// `check_address_allocation_log` (`AllocationLog.c:34-51`). + fn status_of(&self, addr: usize) -> AddressStatus { + match self.allocation.iter().rev().find(|(a, _)| *a == addr) { + Some((_, true)) => AddressStatus::Free, + Some((_, false)) => AddressStatus::Dynamic, + None => AddressStatus::Static, + } + } + + /// `map2check_add_store_pointer` (`AnalysisModeMemcleanup.c:242-291`). + fn store_pointer(&mut self, var: usize, value: usize) { + let status = self.status_of(value); + let redundant = self + .list + .iter() + .rev() + .find(|r| r.var == var) + .is_some_and(|r| r.points_to == value && r.status == status); + if !redundant { + self.list.push(ListRow { + var, + points_to: value, + status, + }); + self.update_reference(value, status); + } + } + + /// `update_reference_list_log` (`AnalysisModeMemcleanup.c:181-240`): for + /// each row that points to `address` with another status, the newest row + /// of the same variable that points to `address` gets a copy with the new + /// status appended. Rows appended here are not revisited. + fn update_reference(&mut self, address: usize, status: AddressStatus) { + let current_size = self.list.len(); + for i in 0..current_size { + let i_row = self.list[i]; + if i_row.points_to != address || i_row.status == status { + continue; + } + if let Some(j_row) = self.list[i..current_size] + .iter() + .rev() + .find(|r| r.points_to == address && r.var == i_row.var) + .copied() + { + if j_row.status != status { + self.list.push(ListRow { status, ..j_row }); + } + } + } + } + + /// `valid_allocation_log` (`AllocationLog.c:69-101`): the LAST address, + /// in allocation-log order, that is not free. Rows are unique per + /// address, so "no later freed row" reduces to the row's own flag. + fn last_leaked_address(&self) -> Option { + self.allocation + .iter() + .filter(|(_, is_free)| !is_free) + .map(|(addr, _)| *addr) + .next_back() + } + + /// `is_memcleanup_error` (`ListLog.c:40-65`). + fn is_memcleanup_error(&self, address: usize) -> bool { + let list = &self.list; + (0..list.len()).rev().any(|i| { + if list[i].points_to != address { + return false; + } + let pointer = list[i].var; + list[i + 1..] + .iter() + .rev() + .find(|r| r.var == pointer) + .is_none_or(|r| r.points_to == address) + }) + } +} + +/// The address `valid_allocation_log` of the Map2Check `develop` C runtime +/// returns for these entries: the last leaked heap address, or `None`. +pub fn last_leaked_address(entries: &[MemTrackEntry]) -> Option { + DevelopLogs::replay(entries).last_leaked_address() +} + +/// `is_memcleanup_error(address, &list_log)` of the Map2Check `develop` C +/// runtime over these entries: some pointer variable's latest stored value +/// still points to `address`. +pub fn is_memcleanup_error(entries: &[MemTrackEntry], address: usize) -> bool { + DevelopLogs::replay(entries).is_memcleanup_error(address) +} + +/// The memcleanup-mode verdict of the Map2Check `develop` C runtime +/// (`AnalysisModeMemcleanup.c:138-148`): the last leaked address is still +/// referenced by a pointer variable. +pub fn has_develop_memcleanup_error(entries: &[MemTrackEntry]) -> bool { + let logs = DevelopLogs::replay(entries); + logs.last_leaked_address() + .is_some_and(|addr| logs.is_memcleanup_error(addr)) +} + /// Checks whether the `size` bytes at `address` lie outside every valid block. /// Replaces `is_a_invalid_address_in_cntr`. /// @@ -199,6 +379,81 @@ pub fn find_by_address(entries: &[MemTrackEntry], address: usize) -> Option<&Mem mod tests { use super::*; + // ── develop's memcleanup rule (AllocationLog.c:69-101, ListLog.c:40-65) ── + + fn alloc(a: usize) -> MemTrackEntry { + make_entry(a, a, true, false, 8) + } + + fn free(a: usize) -> MemTrackEntry { + let mut e = make_entry(a, a, false, false, 0); + e.set_free(); + e + } + + fn store(var: usize, value: usize) -> MemTrackEntry { + MemTrackEntry::new(1, 1, 0, var, value, false, false, "p", "main", 0, 1, false) + } + + #[test] + fn last_leaked_address_is_the_last_live_block_in_allocation_order() { + assert_eq!(last_leaked_address(&[]), None); + assert_eq!(last_leaked_address(&[alloc(0xA0), free(0xA0)]), None); + assert_eq!(last_leaked_address(&[alloc(0xA0), alloc(0xB0)]), Some(0xB0)); + assert_eq!( + last_leaked_address(&[alloc(0xA0), alloc(0xB0), free(0xB0)]), + Some(0xA0) + ); + // develop updates the row of a reused address in place, so the order + // is that of the first allocation of each address. + assert_eq!( + last_leaked_address(&[alloc(0xA0), alloc(0xB0), free(0xA0), alloc(0xA0)]), + Some(0xB0) + ); + // Pointer stores and stack slots are not allocations. + assert_eq!(last_leaked_address(&[store(0x10, 0xA0)]), None); + } + + #[test] + fn memcleanup_error_needs_a_pointer_whose_latest_value_is_the_block() { + let e = [alloc(0xA0), store(0x10, 0xA0)]; + assert!(is_memcleanup_error(&e, 0xA0)); + let e = [alloc(0xA0), store(0x10, 0xA0), store(0x10, 0xB0)]; + assert!(!is_memcleanup_error(&e, 0xA0)); + let e = [alloc(0xA0)]; + assert!(!is_memcleanup_error(&e, 0xA0)); + let e = [ + alloc(0xA0), + store(0x10, 0xA0), + store(0x20, 0xA0), + store(0x10, 0), + ]; + assert!(is_memcleanup_error(&e, 0xA0)); + } + + #[test] + fn memcleanup_redundant_store_is_not_logged_again() { + // add_store_pointer skips a store equal to the variable's latest row + // (AnalysisModeMemcleanup.c:272-283); the verdict is unchanged. + let e = [alloc(0xA0), store(0x10, 0xA0), store(0x10, 0xA0)]; + assert!(is_memcleanup_error(&e, 0xA0)); + } + + #[test] + fn memcleanup_free_re_points_variables_as_update_reference_list_log() { + // AnalysisModeMemcleanup.c:181-240: freeing A appends a FREE copy of + // the latest `p = A` row, which makes `p` point to A again. + let e = [ + alloc(0xA0), + alloc(0xB0), + store(0x10, 0xA0), + store(0x10, 0xB0), + free(0xA0), + ]; + assert!(!is_memcleanup_error(&e, 0xB0)); + assert!(is_memcleanup_error(&e[..4], 0xB0)); + } + fn make_entry( var_addr: usize, points_to: usize, diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index d5b4710..4a95962 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -382,6 +382,32 @@ static int test_analysis_modes(void) { if (get_current_property() != Reachability) { return 4; } + + /* Out-of-range values are ignored: the mode stays Target (I-1). */ + map2check_set_mode(-1); + map2check_set_mode(6); + map2check_set_mode(2147483647); + map2check_init(); + map2check_check_deref((void *)0xdead, 0, 1, "main"); + if (get_current_property() != None) { + return 5; + } + + /* Memcleanup mode: only a leak still referenced by a pointer (I-2). */ + map2check_set_mode(AnalysisMode_Memcleanup); + map2check_init(); + map2check_map_malloc((void *)0x3000, 8); + map2check_check_mem_endprog(); + if (get_current_property() != None) { + return 6; + } + map2check_init(); + map2check_map_malloc((void *)0x3000, 8); + map2check_map_store_pointer((void *)0x10, (void *)0x3000, 0, "p", 1, "main"); + map2check_check_mem_endprog(); + if (get_current_property() != MemsafetyMemcleanup) { + return 7; + } map2check_reset(); return 0; } From 95243dc4e5a5f460e1a38646d8c0201a47c756bb Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 21:49:57 -0400 Subject: [PATCH 22/63] docs: link a witness-enabling constructor into witness binaries in the cutover plan --- docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md index dc1bea2..df984ca 100644 --- a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md +++ b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md @@ -1163,6 +1163,7 @@ TEST(CallerLink, EveryModeAndGeneratorLinksRuntime) { Os enumeradores seguem `modules/frontend/caller.hpp:21-45` de `develop`; incluir também `COVER_BRANCHES_MODE` no laço. - [ ] **Step 2–4:** Ver falhar, extrair as funções e implementar, `ctest --output-on-failure` verde. +- [ ] **Step 4b:** Os binários de witness (`-witness-fuzzed.out` e o `-witness-result.bc` que vai ao KLEE) ligam também um objeto gerado pelo frontend com `__attribute__((constructor)) static void m2c_witness(void) { map2check_enable_witness(); }`, compilado com `clang-16 -c` (nativo) ou `-emit-llvm` (KLEE). Os binários de verificação não o ligam. Isso reproduz a escolha `WitnessGeneration.bc` × `WitnessGenerationNone.bc` de `caller.cpp:635-640` (ruling da Task 2.4). Teste em `CallerLink`: o comando de witness contém o objeto e o de verificação não. - [ ] **Step 5:** Se D3 = remover, adicionar o aviso de `--btree` com o seu teste. - [ ] **Step 6: Commit (na branch)** `feat: link map2check-library instead of the C bitcode runtime` From 1ed268b3cfde393d348919c1f565debda96490a0 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 22:13:01 -0400 Subject: [PATCH 23/63] test: capture result files of the develop C runtime as golden files (#19) Drivers in tests/legacy_golden/drivers call the C runtime of Map2Check develop directly, one scenario per verdict, and build_goldens.sh links them as Caller::linkLLVM does for the witness binary. Every file each run writes is kept under tests/legacy_golden// with its exit status. --- tests/legacy_golden/ASSERT/.exit_status | 1 + tests/legacy_golden/ASSERT/klee_log.csv | 5 + .../ASSERT/map2check_checked_error | 1 + tests/legacy_golden/ASSERT/map2check_property | 3 + .../ASSERT/map2check_property_assert | 3 + tests/legacy_golden/ASSERT/track_bb_log.st | 2 + tests/legacy_golden/FALSE-DEREF/.exit_status | 1 + .../FALSE-DEREF/allocation_log.csv | 1 + tests/legacy_golden/FALSE-DEREF/heap_log.csv | 2 + tests/legacy_golden/FALSE-DEREF/klee_log.csv | 5 + tests/legacy_golden/FALSE-DEREF/list_log.csv | 1 + .../FALSE-DEREF/map2check_checked_error | 1 + .../FALSE-DEREF/map2check_property | 3 + .../FALSE-DEREF/map2check_property_deref | 3 + .../legacy_golden/FALSE-DEREF/track_bb_log.st | 2 + tests/legacy_golden/FALSE-FREE/.exit_status | 1 + .../FALSE-FREE/allocation_log.csv | 1 + tests/legacy_golden/FALSE-FREE/heap_log.csv | 2 + tests/legacy_golden/FALSE-FREE/klee_log.csv | 5 + tests/legacy_golden/FALSE-FREE/list_log.csv | 2 + .../FALSE-FREE/map2check_checked_error | 1 + .../FALSE-FREE/map2check_property | 3 + .../FALSE-FREE/map2check_property_free | 3 + .../legacy_golden/FALSE-FREE/track_bb_log.st | 2 + .../FALSE-MEMCLEANUP/.exit_status | 1 + .../FALSE-MEMCLEANUP/allocation_log.csv | 2 + .../FALSE-MEMCLEANUP/heap_log.csv | 2 + .../FALSE-MEMCLEANUP/klee_log.csv | 5 + .../FALSE-MEMCLEANUP/list_log.csv | 1 + .../FALSE-MEMCLEANUP/map2check_checked_error | 1 + .../FALSE-MEMCLEANUP/map2check_property | 1 + .../map2check_property_memcleanup | 1 + .../FALSE-MEMCLEANUP/track_bb_log.st | 2 + .../legacy_golden/FALSE-MEMTRACK/.exit_status | 1 + .../FALSE-MEMTRACK/allocation_log.csv | 2 + .../legacy_golden/FALSE-MEMTRACK/heap_log.csv | 2 + .../legacy_golden/FALSE-MEMTRACK/klee_log.csv | 5 + .../legacy_golden/FALSE-MEMTRACK/list_log.csv | 1 + .../FALSE-MEMTRACK/map2check_checked_error | 1 + .../FALSE-MEMTRACK/map2check_property | 1 + .../map2check_property_memtrack | 1 + .../FALSE-MEMTRACK/track_bb_log.st | 2 + tests/legacy_golden/NONE/.exit_status | 1 + tests/legacy_golden/NONE/allocation_log.csv | 1 + tests/legacy_golden/NONE/heap_log.csv | 2 + tests/legacy_golden/NONE/list_log.csv | 2 + tests/legacy_golden/NONE/map2check_property | 1 + tests/legacy_golden/NONE/track_bb_log.st | 2 + tests/legacy_golden/OVERFLOW/.exit_status | 1 + tests/legacy_golden/OVERFLOW/klee_log.csv | 5 + .../OVERFLOW/map2check_checked_error | 1 + .../legacy_golden/OVERFLOW/map2check_property | 3 + .../OVERFLOW/map2check_property_overflow | 3 + tests/legacy_golden/OVERFLOW/track_bb_log.st | 2 + tests/legacy_golden/README.md | 68 +++++++++++ .../legacy_golden/TARGET-REACHED/.exit_status | 1 + .../legacy_golden/TARGET-REACHED/klee_log.csv | 5 + .../TARGET-REACHED/map2check_checked_error | 1 + .../TARGET-REACHED/map2check_property | 3 + .../TARGET-REACHED/track_bb_log.st | 2 + tests/legacy_golden/UNKNOWN/.exit_status | 1 + .../legacy_golden/UNKNOWN/allocation_log.csv | 1 + tests/legacy_golden/UNKNOWN/heap_log.csv | 2 + tests/legacy_golden/UNKNOWN/klee_log.csv | 5 + tests/legacy_golden/UNKNOWN/list_log.csv | 1 + .../UNKNOWN/map2check_checked_error | 1 + .../UNKNOWN/map2check_property__unknown | 1 + tests/legacy_golden/UNKNOWN/track_bb_log.st | 2 + tests/legacy_golden/drivers/ASSERT.c | 13 +++ tests/legacy_golden/drivers/FALSE-DEREF.c | 15 +++ tests/legacy_golden/drivers/FALSE-FREE.c | 15 +++ .../legacy_golden/drivers/FALSE-MEMCLEANUP.c | 13 +++ tests/legacy_golden/drivers/FALSE-MEMTRACK.c | 13 +++ tests/legacy_golden/drivers/NONE.c | 14 +++ .../drivers/NonDetGeneratorLog.c | 28 +++++ tests/legacy_golden/drivers/OVERFLOW.c | 13 +++ tests/legacy_golden/drivers/TARGET-REACHED.c | 13 +++ tests/legacy_golden/drivers/UNKNOWN.c | 14 +++ tests/legacy_golden/drivers/build_goldens.sh | 71 ++++++++++++ tests/legacy_golden/drivers/golden.h | 106 ++++++++++++++++++ 80 files changed, 532 insertions(+) create mode 100644 tests/legacy_golden/ASSERT/.exit_status create mode 100644 tests/legacy_golden/ASSERT/klee_log.csv create mode 100644 tests/legacy_golden/ASSERT/map2check_checked_error create mode 100644 tests/legacy_golden/ASSERT/map2check_property create mode 100644 tests/legacy_golden/ASSERT/map2check_property_assert create mode 100644 tests/legacy_golden/ASSERT/track_bb_log.st create mode 100644 tests/legacy_golden/FALSE-DEREF/.exit_status create mode 100644 tests/legacy_golden/FALSE-DEREF/allocation_log.csv create mode 100644 tests/legacy_golden/FALSE-DEREF/heap_log.csv create mode 100644 tests/legacy_golden/FALSE-DEREF/klee_log.csv create mode 100644 tests/legacy_golden/FALSE-DEREF/list_log.csv create mode 100644 tests/legacy_golden/FALSE-DEREF/map2check_checked_error create mode 100644 tests/legacy_golden/FALSE-DEREF/map2check_property create mode 100644 tests/legacy_golden/FALSE-DEREF/map2check_property_deref create mode 100644 tests/legacy_golden/FALSE-DEREF/track_bb_log.st create mode 100644 tests/legacy_golden/FALSE-FREE/.exit_status create mode 100644 tests/legacy_golden/FALSE-FREE/allocation_log.csv create mode 100644 tests/legacy_golden/FALSE-FREE/heap_log.csv create mode 100644 tests/legacy_golden/FALSE-FREE/klee_log.csv create mode 100644 tests/legacy_golden/FALSE-FREE/list_log.csv create mode 100644 tests/legacy_golden/FALSE-FREE/map2check_checked_error create mode 100644 tests/legacy_golden/FALSE-FREE/map2check_property create mode 100644 tests/legacy_golden/FALSE-FREE/map2check_property_free create mode 100644 tests/legacy_golden/FALSE-FREE/track_bb_log.st create mode 100644 tests/legacy_golden/FALSE-MEMCLEANUP/.exit_status create mode 100644 tests/legacy_golden/FALSE-MEMCLEANUP/allocation_log.csv create mode 100644 tests/legacy_golden/FALSE-MEMCLEANUP/heap_log.csv create mode 100644 tests/legacy_golden/FALSE-MEMCLEANUP/klee_log.csv create mode 100644 tests/legacy_golden/FALSE-MEMCLEANUP/list_log.csv create mode 100644 tests/legacy_golden/FALSE-MEMCLEANUP/map2check_checked_error create mode 100644 tests/legacy_golden/FALSE-MEMCLEANUP/map2check_property create mode 100644 tests/legacy_golden/FALSE-MEMCLEANUP/map2check_property_memcleanup create mode 100644 tests/legacy_golden/FALSE-MEMCLEANUP/track_bb_log.st create mode 100644 tests/legacy_golden/FALSE-MEMTRACK/.exit_status create mode 100644 tests/legacy_golden/FALSE-MEMTRACK/allocation_log.csv create mode 100644 tests/legacy_golden/FALSE-MEMTRACK/heap_log.csv create mode 100644 tests/legacy_golden/FALSE-MEMTRACK/klee_log.csv create mode 100644 tests/legacy_golden/FALSE-MEMTRACK/list_log.csv create mode 100644 tests/legacy_golden/FALSE-MEMTRACK/map2check_checked_error create mode 100644 tests/legacy_golden/FALSE-MEMTRACK/map2check_property create mode 100644 tests/legacy_golden/FALSE-MEMTRACK/map2check_property_memtrack create mode 100644 tests/legacy_golden/FALSE-MEMTRACK/track_bb_log.st create mode 100644 tests/legacy_golden/NONE/.exit_status create mode 100644 tests/legacy_golden/NONE/allocation_log.csv create mode 100644 tests/legacy_golden/NONE/heap_log.csv create mode 100644 tests/legacy_golden/NONE/list_log.csv create mode 100644 tests/legacy_golden/NONE/map2check_property create mode 100644 tests/legacy_golden/NONE/track_bb_log.st create mode 100644 tests/legacy_golden/OVERFLOW/.exit_status create mode 100644 tests/legacy_golden/OVERFLOW/klee_log.csv create mode 100644 tests/legacy_golden/OVERFLOW/map2check_checked_error create mode 100644 tests/legacy_golden/OVERFLOW/map2check_property create mode 100644 tests/legacy_golden/OVERFLOW/map2check_property_overflow create mode 100644 tests/legacy_golden/OVERFLOW/track_bb_log.st create mode 100644 tests/legacy_golden/README.md create mode 100644 tests/legacy_golden/TARGET-REACHED/.exit_status create mode 100644 tests/legacy_golden/TARGET-REACHED/klee_log.csv create mode 100644 tests/legacy_golden/TARGET-REACHED/map2check_checked_error create mode 100644 tests/legacy_golden/TARGET-REACHED/map2check_property create mode 100644 tests/legacy_golden/TARGET-REACHED/track_bb_log.st create mode 100644 tests/legacy_golden/UNKNOWN/.exit_status create mode 100644 tests/legacy_golden/UNKNOWN/allocation_log.csv create mode 100644 tests/legacy_golden/UNKNOWN/heap_log.csv create mode 100644 tests/legacy_golden/UNKNOWN/klee_log.csv create mode 100644 tests/legacy_golden/UNKNOWN/list_log.csv create mode 100644 tests/legacy_golden/UNKNOWN/map2check_checked_error create mode 100644 tests/legacy_golden/UNKNOWN/map2check_property__unknown create mode 100644 tests/legacy_golden/UNKNOWN/track_bb_log.st create mode 100644 tests/legacy_golden/drivers/ASSERT.c create mode 100644 tests/legacy_golden/drivers/FALSE-DEREF.c create mode 100644 tests/legacy_golden/drivers/FALSE-FREE.c create mode 100644 tests/legacy_golden/drivers/FALSE-MEMCLEANUP.c create mode 100644 tests/legacy_golden/drivers/FALSE-MEMTRACK.c create mode 100644 tests/legacy_golden/drivers/NONE.c create mode 100644 tests/legacy_golden/drivers/NonDetGeneratorLog.c create mode 100644 tests/legacy_golden/drivers/OVERFLOW.c create mode 100644 tests/legacy_golden/drivers/TARGET-REACHED.c create mode 100644 tests/legacy_golden/drivers/UNKNOWN.c create mode 100755 tests/legacy_golden/drivers/build_goldens.sh create mode 100644 tests/legacy_golden/drivers/golden.h diff --git a/tests/legacy_golden/ASSERT/.exit_status b/tests/legacy_golden/ASSERT/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/ASSERT/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/ASSERT/klee_log.csv b/tests/legacy_golden/ASSERT/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/ASSERT/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/ASSERT/map2check_checked_error b/tests/legacy_golden/ASSERT/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/ASSERT/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/ASSERT/map2check_property b/tests/legacy_golden/ASSERT/map2check_property new file mode 100644 index 0000000..2c952a8 --- /dev/null +++ b/tests/legacy_golden/ASSERT/map2check_property @@ -0,0 +1,3 @@ +ASSERT +Line: 21 +Function: main diff --git a/tests/legacy_golden/ASSERT/map2check_property_assert b/tests/legacy_golden/ASSERT/map2check_property_assert new file mode 100644 index 0000000..2c952a8 --- /dev/null +++ b/tests/legacy_golden/ASSERT/map2check_property_assert @@ -0,0 +1,3 @@ +ASSERT +Line: 21 +Function: main diff --git a/tests/legacy_golden/ASSERT/track_bb_log.st b/tests/legacy_golden/ASSERT/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/ASSERT/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/FALSE-DEREF/.exit_status b/tests/legacy_golden/FALSE-DEREF/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/FALSE-DEREF/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/FALSE-DEREF/allocation_log.csv b/tests/legacy_golden/FALSE-DEREF/allocation_log.csv new file mode 100644 index 0000000..7da6232 --- /dev/null +++ b/tests/legacy_golden/FALSE-DEREF/allocation_log.csv @@ -0,0 +1 @@ +0x10000100;16;0 diff --git a/tests/legacy_golden/FALSE-DEREF/heap_log.csv b/tests/legacy_golden/FALSE-DEREF/heap_log.csv new file mode 100644 index 0000000..b4ac048 --- /dev/null +++ b/tests/legacy_golden/FALSE-DEREF/heap_log.csv @@ -0,0 +1,2 @@ +0x10000000;p;8 +0x10000040;main;1 diff --git a/tests/legacy_golden/FALSE-DEREF/klee_log.csv b/tests/legacy_golden/FALSE-DEREF/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/FALSE-DEREF/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/FALSE-DEREF/list_log.csv b/tests/legacy_golden/FALSE-DEREF/list_log.csv new file mode 100644 index 0000000..8c36438 --- /dev/null +++ b/tests/legacy_golden/FALSE-DEREF/list_log.csv @@ -0,0 +1 @@ +0;0x10000000;0x10000100;1;0;1;p;11;main;5 diff --git a/tests/legacy_golden/FALSE-DEREF/map2check_checked_error b/tests/legacy_golden/FALSE-DEREF/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/FALSE-DEREF/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/FALSE-DEREF/map2check_property b/tests/legacy_golden/FALSE-DEREF/map2check_property new file mode 100644 index 0000000..fa32018 --- /dev/null +++ b/tests/legacy_golden/FALSE-DEREF/map2check_property @@ -0,0 +1,3 @@ +FALSE-DEREF +Line: 14 +Function: main diff --git a/tests/legacy_golden/FALSE-DEREF/map2check_property_deref b/tests/legacy_golden/FALSE-DEREF/map2check_property_deref new file mode 100644 index 0000000..fa32018 --- /dev/null +++ b/tests/legacy_golden/FALSE-DEREF/map2check_property_deref @@ -0,0 +1,3 @@ +FALSE-DEREF +Line: 14 +Function: main diff --git a/tests/legacy_golden/FALSE-DEREF/track_bb_log.st b/tests/legacy_golden/FALSE-DEREF/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/FALSE-DEREF/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/FALSE-FREE/.exit_status b/tests/legacy_golden/FALSE-FREE/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/FALSE-FREE/allocation_log.csv b/tests/legacy_golden/FALSE-FREE/allocation_log.csv new file mode 100644 index 0000000..1d1e507 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE/allocation_log.csv @@ -0,0 +1 @@ +0x10000100;0;1 diff --git a/tests/legacy_golden/FALSE-FREE/heap_log.csv b/tests/legacy_golden/FALSE-FREE/heap_log.csv new file mode 100644 index 0000000..b4ac048 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE/heap_log.csv @@ -0,0 +1,2 @@ +0x10000000;p;8 +0x10000040;main;1 diff --git a/tests/legacy_golden/FALSE-FREE/klee_log.csv b/tests/legacy_golden/FALSE-FREE/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/FALSE-FREE/list_log.csv b/tests/legacy_golden/FALSE-FREE/list_log.csv new file mode 100644 index 0000000..645df4d --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE/list_log.csv @@ -0,0 +1,2 @@ +0;0x10000000;0x10000100;1;0;1;p;11;main;5 +0;0x10000000;0x10000100;1;1;0;p;12;main;6 diff --git a/tests/legacy_golden/FALSE-FREE/map2check_checked_error b/tests/legacy_golden/FALSE-FREE/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/FALSE-FREE/map2check_property b/tests/legacy_golden/FALSE-FREE/map2check_property new file mode 100644 index 0000000..138db76 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE/map2check_property @@ -0,0 +1,3 @@ +FALSE-FREE +Line: 13 +Function: main diff --git a/tests/legacy_golden/FALSE-FREE/map2check_property_free b/tests/legacy_golden/FALSE-FREE/map2check_property_free new file mode 100644 index 0000000..138db76 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE/map2check_property_free @@ -0,0 +1,3 @@ +FALSE-FREE +Line: 13 +Function: main diff --git a/tests/legacy_golden/FALSE-FREE/track_bb_log.st b/tests/legacy_golden/FALSE-FREE/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/FALSE-MEMCLEANUP/.exit_status b/tests/legacy_golden/FALSE-MEMCLEANUP/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMCLEANUP/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/FALSE-MEMCLEANUP/allocation_log.csv b/tests/legacy_golden/FALSE-MEMCLEANUP/allocation_log.csv new file mode 100644 index 0000000..644ff49 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMCLEANUP/allocation_log.csv @@ -0,0 +1,2 @@ +0x10000100;16;0 +0x10000100;0;1 diff --git a/tests/legacy_golden/FALSE-MEMCLEANUP/heap_log.csv b/tests/legacy_golden/FALSE-MEMCLEANUP/heap_log.csv new file mode 100644 index 0000000..b4ac048 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMCLEANUP/heap_log.csv @@ -0,0 +1,2 @@ +0x10000000;p;8 +0x10000040;main;1 diff --git a/tests/legacy_golden/FALSE-MEMCLEANUP/klee_log.csv b/tests/legacy_golden/FALSE-MEMCLEANUP/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMCLEANUP/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/FALSE-MEMCLEANUP/list_log.csv b/tests/legacy_golden/FALSE-MEMCLEANUP/list_log.csv new file mode 100644 index 0000000..8c36438 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMCLEANUP/list_log.csv @@ -0,0 +1 @@ +0;0x10000000;0x10000100;1;0;1;p;11;main;5 diff --git a/tests/legacy_golden/FALSE-MEMCLEANUP/map2check_checked_error b/tests/legacy_golden/FALSE-MEMCLEANUP/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMCLEANUP/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/FALSE-MEMCLEANUP/map2check_property b/tests/legacy_golden/FALSE-MEMCLEANUP/map2check_property new file mode 100644 index 0000000..76ec98a --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMCLEANUP/map2check_property @@ -0,0 +1 @@ +FALSE-MEMCLEANUP diff --git a/tests/legacy_golden/FALSE-MEMCLEANUP/map2check_property_memcleanup b/tests/legacy_golden/FALSE-MEMCLEANUP/map2check_property_memcleanup new file mode 100644 index 0000000..76ec98a --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMCLEANUP/map2check_property_memcleanup @@ -0,0 +1 @@ +FALSE-MEMCLEANUP diff --git a/tests/legacy_golden/FALSE-MEMCLEANUP/track_bb_log.st b/tests/legacy_golden/FALSE-MEMCLEANUP/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMCLEANUP/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/FALSE-MEMTRACK/.exit_status b/tests/legacy_golden/FALSE-MEMTRACK/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMTRACK/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/FALSE-MEMTRACK/allocation_log.csv b/tests/legacy_golden/FALSE-MEMTRACK/allocation_log.csv new file mode 100644 index 0000000..644ff49 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMTRACK/allocation_log.csv @@ -0,0 +1,2 @@ +0x10000100;16;0 +0x10000100;0;1 diff --git a/tests/legacy_golden/FALSE-MEMTRACK/heap_log.csv b/tests/legacy_golden/FALSE-MEMTRACK/heap_log.csv new file mode 100644 index 0000000..b4ac048 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMTRACK/heap_log.csv @@ -0,0 +1,2 @@ +0x10000000;p;8 +0x10000040;main;1 diff --git a/tests/legacy_golden/FALSE-MEMTRACK/klee_log.csv b/tests/legacy_golden/FALSE-MEMTRACK/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMTRACK/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/FALSE-MEMTRACK/list_log.csv b/tests/legacy_golden/FALSE-MEMTRACK/list_log.csv new file mode 100644 index 0000000..8c36438 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMTRACK/list_log.csv @@ -0,0 +1 @@ +0;0x10000000;0x10000100;1;0;1;p;11;main;5 diff --git a/tests/legacy_golden/FALSE-MEMTRACK/map2check_checked_error b/tests/legacy_golden/FALSE-MEMTRACK/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMTRACK/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/FALSE-MEMTRACK/map2check_property b/tests/legacy_golden/FALSE-MEMTRACK/map2check_property new file mode 100644 index 0000000..32e1ea0 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMTRACK/map2check_property @@ -0,0 +1 @@ +FALSE-MEMTRACK diff --git a/tests/legacy_golden/FALSE-MEMTRACK/map2check_property_memtrack b/tests/legacy_golden/FALSE-MEMTRACK/map2check_property_memtrack new file mode 100644 index 0000000..32e1ea0 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMTRACK/map2check_property_memtrack @@ -0,0 +1 @@ +FALSE-MEMTRACK diff --git a/tests/legacy_golden/FALSE-MEMTRACK/track_bb_log.st b/tests/legacy_golden/FALSE-MEMTRACK/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/FALSE-MEMTRACK/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/NONE/.exit_status b/tests/legacy_golden/NONE/.exit_status new file mode 100644 index 0000000..573541a --- /dev/null +++ b/tests/legacy_golden/NONE/.exit_status @@ -0,0 +1 @@ +0 diff --git a/tests/legacy_golden/NONE/allocation_log.csv b/tests/legacy_golden/NONE/allocation_log.csv new file mode 100644 index 0000000..1d1e507 --- /dev/null +++ b/tests/legacy_golden/NONE/allocation_log.csv @@ -0,0 +1 @@ +0x10000100;0;1 diff --git a/tests/legacy_golden/NONE/heap_log.csv b/tests/legacy_golden/NONE/heap_log.csv new file mode 100644 index 0000000..b4ac048 --- /dev/null +++ b/tests/legacy_golden/NONE/heap_log.csv @@ -0,0 +1,2 @@ +0x10000000;p;8 +0x10000040;main;1 diff --git a/tests/legacy_golden/NONE/list_log.csv b/tests/legacy_golden/NONE/list_log.csv new file mode 100644 index 0000000..645df4d --- /dev/null +++ b/tests/legacy_golden/NONE/list_log.csv @@ -0,0 +1,2 @@ +0;0x10000000;0x10000100;1;0;1;p;11;main;5 +0;0x10000000;0x10000100;1;1;0;p;12;main;6 diff --git a/tests/legacy_golden/NONE/map2check_property b/tests/legacy_golden/NONE/map2check_property new file mode 100644 index 0000000..7a2d820 --- /dev/null +++ b/tests/legacy_golden/NONE/map2check_property @@ -0,0 +1 @@ +NONE diff --git a/tests/legacy_golden/NONE/track_bb_log.st b/tests/legacy_golden/NONE/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/NONE/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/OVERFLOW/.exit_status b/tests/legacy_golden/OVERFLOW/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/OVERFLOW/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/OVERFLOW/klee_log.csv b/tests/legacy_golden/OVERFLOW/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/OVERFLOW/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/OVERFLOW/map2check_checked_error b/tests/legacy_golden/OVERFLOW/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/OVERFLOW/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/OVERFLOW/map2check_property b/tests/legacy_golden/OVERFLOW/map2check_property new file mode 100644 index 0000000..d593013 --- /dev/null +++ b/tests/legacy_golden/OVERFLOW/map2check_property @@ -0,0 +1,3 @@ +OVERFLOW +Line: 20 +Function: main diff --git a/tests/legacy_golden/OVERFLOW/map2check_property_overflow b/tests/legacy_golden/OVERFLOW/map2check_property_overflow new file mode 100644 index 0000000..d593013 --- /dev/null +++ b/tests/legacy_golden/OVERFLOW/map2check_property_overflow @@ -0,0 +1,3 @@ +OVERFLOW +Line: 20 +Function: main diff --git a/tests/legacy_golden/OVERFLOW/track_bb_log.st b/tests/legacy_golden/OVERFLOW/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/OVERFLOW/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/README.md b/tests/legacy_golden/README.md new file mode 100644 index 0000000..050df4a --- /dev/null +++ b/tests/legacy_golden/README.md @@ -0,0 +1,68 @@ +# Golden files of the C runtime of Map2Check `develop` + +Each `/` directory holds every file that the C runtime of Map2Check +`develop` (`modules/backend/library/lib/`) writes for one scenario, plus +`.exit_status` (134 = `abort()`, 0 = normal exit). +`rust/src/ffi_golden_tests.rs` makes the same calls through this library, +with the mode set and witness on, in a child process, and compares every +file byte for byte (#19). + +The Map2Check frontend deletes its scratch directory, so the files cannot be +copied out of a normal run. Instead `drivers/.c` calls the C runtime +API directly, with the calls an instrumented program makes, and +`drivers/build_goldens.sh` builds each driver as `Caller::linkLLVM` does +(`modules/frontend/caller.cpp:550-640`): the driver bitcode, then +`Map2CheckFunctions`, `TrackBBLog`, `NonDetLog`, `PropertyGenerator`, +`ContainerRealloc`, the `AnalysisMode*.c` of the tag (and `AllocationLog`, +`ListLog`, `HeapLog` for the memory modes), the nondet generator and +`WitnessGeneration` (the witness binary), linked with `llvm-link-16` and +compiled with `clang-16`. Each binary runs in a fresh directory. + +| Tag | Mode file | Calls after `map2check_init(0)` and the shared tracking calls | +|---|---|---| +| `FALSE-FREE` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_free` of `p`, `map2check_free_resolved_address` of the same block | +| `FALSE-DEREF` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_load` of an unallocated address, `map2check_check_deref` | +| `FALSE-MEMTRACK` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_success` (the block leaks) | +| `FALSE-MEMCLEANUP` | `AnalysisModeMemcleanup.c` | memory prelude, `map2check_success` (the block leaks, `p` points to it) | +| `NONE` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_free` of `p`, `map2check_success` | +| `UNKNOWN` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_malloc(NULL, 4)` | +| `OVERFLOW` | `AnalysisModeOverflow.c` | `map2check_binop_add(INT_MAX, 1, 32, ...)` | +| `ASSERT` | `AnalysisModeAssert.c` | `map2check_assert(0, ...)` | +| `TARGET-REACHED` | `AnalysisModeNone.c` (`REACHABILITY_MODE`) | `map2check_target_function` | + +`drivers/golden.h` holds the shared calls: three basic blocks (one line +repeated), five nondet values (int, uint, char, long, double) and the +memory prelude (`p` on the stack, `&main`, `p = malloc(16)`). + +Two accommodations make the output reproducible and complete: + +- **Fixed addresses.** Every address the runtime sees lies in an arena + mapped at `0x10000000` (`MAP_FIXED_NOREPLACE`), so the `%p` columns are + the same on every run and the Rust test can pass the same numbers. The + runtime calls `free()` on a leaked block at exit (`AllocationLog.c:91`); + the link uses `-Wl,--wrap=free` and `__wrap_free` ignores arena addresses. +- **Nondet log.** `NonDetGeneratorNone.c` writes no `klee_log.csv` (its + `nondet_generate_aux_witness_files` is empty, `:42`), so the drivers link + `drivers/NonDetGeneratorLog.c`: the `nondet_init`, `nondet_destroy` and + `nondet_generate_aux_witness_files` of `NonDetGeneratorAFL.c` (`:25`, + `:27`, `:39-41`) with the plain `main` of `NonDetGeneratorNone.c`. + +## Regenerating + +From the repository root, with the develop checkout of Map2Check at +`$MAP2CHECK` (here `/home/rock/Documents/Dev/Map2Check-cutover`, branch +`test/cutover-baseline` at `b3c10cb7`, whose `modules/backend/library` and +`caller.cpp` are those of `f01e8c3e`, the commit of the image), mounting +read-only what is only read: + +```sh +docker run --rm --user "$(id -u):$(id -g)" \ + -v "$MAP2CHECK/modules/backend/library":/runtime:ro \ + -v "$PWD/tests/legacy_golden/drivers":/drivers:ro \ + -v "$PWD/tests/legacy_golden":/out \ + map2check-dev:cutover-f01e8c3e bash /drivers/build_goldens.sh +``` + +The script prints, per tag, the exit status and the files written. The +runtime compiles with warnings (`map2check_posix` prototypes); they are +expected. diff --git a/tests/legacy_golden/TARGET-REACHED/.exit_status b/tests/legacy_golden/TARGET-REACHED/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/TARGET-REACHED/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/TARGET-REACHED/klee_log.csv b/tests/legacy_golden/TARGET-REACHED/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/TARGET-REACHED/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/TARGET-REACHED/map2check_checked_error b/tests/legacy_golden/TARGET-REACHED/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/TARGET-REACHED/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/TARGET-REACHED/map2check_property b/tests/legacy_golden/TARGET-REACHED/map2check_property new file mode 100644 index 0000000..5df52c6 --- /dev/null +++ b/tests/legacy_golden/TARGET-REACHED/map2check_property @@ -0,0 +1,3 @@ +TARGET-REACHED +Line: 22 +Function: reach_error diff --git a/tests/legacy_golden/TARGET-REACHED/track_bb_log.st b/tests/legacy_golden/TARGET-REACHED/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/TARGET-REACHED/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/UNKNOWN/.exit_status b/tests/legacy_golden/UNKNOWN/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/UNKNOWN/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/UNKNOWN/allocation_log.csv b/tests/legacy_golden/UNKNOWN/allocation_log.csv new file mode 100644 index 0000000..7da6232 --- /dev/null +++ b/tests/legacy_golden/UNKNOWN/allocation_log.csv @@ -0,0 +1 @@ +0x10000100;16;0 diff --git a/tests/legacy_golden/UNKNOWN/heap_log.csv b/tests/legacy_golden/UNKNOWN/heap_log.csv new file mode 100644 index 0000000..b4ac048 --- /dev/null +++ b/tests/legacy_golden/UNKNOWN/heap_log.csv @@ -0,0 +1,2 @@ +0x10000000;p;8 +0x10000040;main;1 diff --git a/tests/legacy_golden/UNKNOWN/klee_log.csv b/tests/legacy_golden/UNKNOWN/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/UNKNOWN/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/UNKNOWN/list_log.csv b/tests/legacy_golden/UNKNOWN/list_log.csv new file mode 100644 index 0000000..8c36438 --- /dev/null +++ b/tests/legacy_golden/UNKNOWN/list_log.csv @@ -0,0 +1 @@ +0;0x10000000;0x10000100;1;0;1;p;11;main;5 diff --git a/tests/legacy_golden/UNKNOWN/map2check_checked_error b/tests/legacy_golden/UNKNOWN/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/UNKNOWN/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/UNKNOWN/map2check_property__unknown b/tests/legacy_golden/UNKNOWN/map2check_property__unknown new file mode 100644 index 0000000..359456b --- /dev/null +++ b/tests/legacy_golden/UNKNOWN/map2check_property__unknown @@ -0,0 +1 @@ +UNKNOWN diff --git a/tests/legacy_golden/UNKNOWN/track_bb_log.st b/tests/legacy_golden/UNKNOWN/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/UNKNOWN/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/drivers/ASSERT.c b/tests/legacy_golden/drivers/ASSERT.c new file mode 100644 index 0000000..c69b2d6 --- /dev/null +++ b/tests/legacy_golden/drivers/ASSERT.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Assert mode: a failed assert. + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + map2check_assert(0, 21, "main"); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/FALSE-DEREF.c b/tests/legacy_golden/drivers/FALSE-DEREF.c new file mode 100644 index 0000000..d22443f --- /dev/null +++ b/tests/legacy_golden/drivers/FALSE-DEREF.c @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Memtrack mode: a 4-byte load from an address that was never allocated. + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + golden_memory(); + map2check_load(WILD, 4); + map2check_check_deref(14, "main"); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/FALSE-FREE.c b/tests/legacy_golden/drivers/FALSE-FREE.c new file mode 100644 index 0000000..cee876b --- /dev/null +++ b/tests/legacy_golden/drivers/FALSE-FREE.c @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Memtrack mode: free(p), then a second free of the same block. + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + golden_memory(); + map2check_free("p", SLOT_P, 1, 12, "main"); + map2check_free_resolved_address(BLOCK, 13, "main", 0); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/FALSE-MEMCLEANUP.c b/tests/legacy_golden/drivers/FALSE-MEMCLEANUP.c new file mode 100644 index 0000000..cd57be2 --- /dev/null +++ b/tests/legacy_golden/drivers/FALSE-MEMCLEANUP.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Memcleanup mode: the block is still allocated and p still points to it. + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + golden_memory(); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/FALSE-MEMTRACK.c b/tests/legacy_golden/drivers/FALSE-MEMTRACK.c new file mode 100644 index 0000000..012d168 --- /dev/null +++ b/tests/legacy_golden/drivers/FALSE-MEMTRACK.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Memtrack mode: the block is still allocated at exit. + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + golden_memory(); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/NONE.c b/tests/legacy_golden/drivers/NONE.c new file mode 100644 index 0000000..73e6386 --- /dev/null +++ b/tests/legacy_golden/drivers/NONE.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Memtrack mode: free(p), then a normal exit. + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + golden_memory(); + map2check_free("p", SLOT_P, 1, 12, "main"); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/NonDetGeneratorLog.c b/tests/legacy_golden/drivers/NonDetGeneratorLog.c new file mode 100644 index 0000000..3702295 --- /dev/null +++ b/tests/legacy_golden/drivers/NonDetGeneratorLog.c @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * + * Nondet generator for the golden drivers. NonDetGeneratorNone.c writes no + * klee_log.csv (its nondet_generate_aux_witness_files is empty, :42), so the + * drivers use this one instead: nondet_init, nondet_destroy and + * nondet_generate_aux_witness_files are those of NonDetGeneratorAFL.c + * (:25, :27, :39-41), and main is that of NonDetGeneratorNone.c (:16-19), + * without the AFL++ persistent loop. + */ +#include "NonDetGenerator.h" +#include "NonDetLog.h" + +extern int __map2check_main__(void); + +int main(void) { + __map2check_main__(); + return 0; +} + +void nondet_init() { nondet_log_init(); } + +void nondet_destroy() { nondet_log_destroy(); } + +void nondet_assume(int expr) { (void)expr; } + +void nondet_generate_aux_witness_files() { + nondet_log_to_file(map2check_nondet_get_log()); +} diff --git a/tests/legacy_golden/drivers/OVERFLOW.c b/tests/legacy_golden/drivers/OVERFLOW.c new file mode 100644 index 0000000..579cef2 --- /dev/null +++ b/tests/legacy_golden/drivers/OVERFLOW.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Overflow mode: INT_MAX + 1 on a 32-bit int. + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + map2check_binop_add(INT_MAX, 1, 32, 20, 0, "main"); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/TARGET-REACHED.c b/tests/legacy_golden/drivers/TARGET-REACHED.c new file mode 100644 index 0000000..f0b8ff9 --- /dev/null +++ b/tests/legacy_golden/drivers/TARGET-REACHED.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Reachability mode (AnalysisModeNone.c): the target function is reached. + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + map2check_target_function("reach_error", 0, 22); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/UNKNOWN.c b/tests/legacy_golden/drivers/UNKNOWN.c new file mode 100644 index 0000000..bb86b54 --- /dev/null +++ b/tests/legacy_golden/drivers/UNKNOWN.c @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Memtrack mode: malloc returns NULL (AnalysisModeMemtrack.c:305-311). + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + golden_memory(); + map2check_malloc(NULL, 4); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/build_goldens.sh b/tests/legacy_golden/drivers/build_goldens.sh new file mode 100755 index 0000000..fb84261 --- /dev/null +++ b/tests/legacy_golden/drivers/build_goldens.sh @@ -0,0 +1,71 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: GPL-3.0-only +# +# Builds each driver against the C runtime of Map2Check develop, as +# Caller::linkLLVM does (modules/frontend/caller.cpp:550-640, witness +# binary: WitnessGeneration.bc), runs it in a fresh directory and copies +# every file it writes to $OUT//. Run inside map2check-dev (see +# ../README.md): +# +# RUNTIME develop's modules/backend/library (read-only) +# DRIVERS this directory (read-only) +# OUT tests/legacy_golden (written) +set -euo pipefail + +RUNTIME=${RUNTIME:-/runtime} +DRIVERS=${DRIVERS:-/drivers} +OUT=${OUT:-/out} +CC=${CC:-clang-16} +LINK=${LINK:-llvm-link-16} +WORK=$(mktemp -d) + +bc() { "$CC" -c -emit-llvm -I"$RUNTIME/header" "$1" -o "$2"; } + +# Files that linkLLVM links in every run (:560-564), the Array container +# (:567-569), the witness generation (:627) and the nondet generator. +for f in Map2CheckFunctions TrackBBLog NonDetLog PropertyGenerator \ + ContainerRealloc WitnessGeneration AnalysisModeMemtrack \ + AnalysisModeMemcleanup AnalysisModeOverflow AnalysisModeAssert \ + AnalysisModeNone AllocationLog ListLog HeapLog; do + bc "$RUNTIME/lib/$f.c" "$WORK/$f.bc" +done +bc "$DRIVERS/NonDetGeneratorLog.c" "$WORK/NonDetGeneratorLog.bc" + +COMMON="Map2CheckFunctions TrackBBLog NonDetLog PropertyGenerator ContainerRealloc" +MEMORY="AllocationLog ListLog HeapLog" + +# tag -> analysis mode files (:578-612) +declare -A MODE=( + [FALSE-FREE]="AnalysisModeMemtrack $MEMORY" + [FALSE-DEREF]="AnalysisModeMemtrack $MEMORY" + [FALSE-MEMTRACK]="AnalysisModeMemtrack $MEMORY" + [FALSE-MEMCLEANUP]="AnalysisModeMemcleanup $MEMORY" + [NONE]="AnalysisModeMemtrack $MEMORY" + [UNKNOWN]="AnalysisModeMemtrack $MEMORY" + [OVERFLOW]="AnalysisModeOverflow" + [ASSERT]="AnalysisModeAssert" + [TARGET-REACHED]="AnalysisModeNone" +) + +for tag in FALSE-FREE FALSE-DEREF FALSE-MEMTRACK FALSE-MEMCLEANUP NONE \ + UNKNOWN OVERFLOW ASSERT TARGET-REACHED; do + bc "$DRIVERS/$tag.c" "$WORK/$tag-output.bc" + inputs=("$WORK/$tag-output.bc") + for f in $COMMON ${MODE[$tag]} NonDetGeneratorLog WitnessGeneration; do + inputs+=("$WORK/$f.bc") + done + "$LINK" "${inputs[@]}" -o "$WORK/$tag-witness-result.bc" + "$CC" "$WORK/$tag-witness-result.bc" -Wl,--wrap=free -o "$WORK/$tag" + + run="$WORK/run-$tag" + mkdir "$run" + status=0 + (cd "$run" && "$WORK/$tag" >stdout.txt 2>stderr.txt) || status=$? + echo "$tag: exit status $status; files: $(ls "$run" | grep -v -e stdout.txt -e stderr.txt | tr '\n' ' ')" + mkdir -p "$OUT/$tag" + for f in "$run"/*; do + case "$(basename "$f")" in stdout.txt | stderr.txt) continue ;; esac + cp "$f" "$OUT/$tag/" + done + echo "$status" >"$OUT/$tag/.exit_status" +done diff --git a/tests/legacy_golden/drivers/golden.h b/tests/legacy_golden/drivers/golden.h new file mode 100644 index 0000000..4077b6a --- /dev/null +++ b/tests/legacy_golden/drivers/golden.h @@ -0,0 +1,106 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * + * Shared scenario for the golden-file drivers. Each driver plays the calls + * that an instrumented program makes on the C runtime of Map2Check develop + * (modules/backend/library/lib), so the files that runtime writes can be + * compared byte for byte with the files written by this library for the + * same calls (rust/src/ffi_golden_tests.rs). + * + * Every address the runtime sees lives in an arena mapped at a fixed + * address, so the %p columns of the logs are the same on every run. The + * runtime never dereferences a heap block; it only reads the pointer slot + * in map2check_free and calls free() on a leaked block at exit + * (AllocationLog.c:91). The build links with -Wl,--wrap=free and + * __wrap_free below ignores arena addresses. + */ +#ifndef GOLDEN_H +#define GOLDEN_H + +#include +#include +#include +#include +#include + +/* develop runtime entry points, as the passes declare them. */ +void map2check_init(int isSvComp); +void map2check_success(void); +void map2check_track_bb(unsigned line, const char *function_name); +void map2check_target_function(const char *func_name, int scope, int line); +void map2check_nondet_int(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_uint(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_char(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_long(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_double(unsigned line, unsigned scope, double value, + const char *function_name); +void map2check_alloca(const char *name, void *ptr, int size, + int size_of_primitive, int line, int scope); +void map2check_function(const char *name, void *ptr); +void map2check_malloc(void *ptr, int size); +void map2check_add_store_pointer(void *var, void *value, unsigned scope, + const char *name, int line, + const char *function_name); +void map2check_free(const char *name, void *ptr, unsigned scope, unsigned line, + const char *function_name); +void map2check_free_resolved_address(void *ptr, unsigned line, + const char *function_name, + short int isNullValid); +void map2check_load(void *ptr, int size); +void map2check_check_deref(int line, const char *function_name); +void map2check_binop_add(long long param1, long long param2, unsigned width, + unsigned line, unsigned scope, char *function_name); +void map2check_assert(int condition, int line, const char *func_name); + +/* Fixed addresses, mirrored by rust/src/output/golden_tests.rs. */ +#define GOLDEN_ARENA 0x10000000UL +#define GOLDEN_ARENA_SIZE 4096UL +#define SLOT_P ((void *)(GOLDEN_ARENA + 0x0)) /* the pointer variable p */ +#define FN_MAIN ((void *)(GOLDEN_ARENA + 0x40)) /* address of main */ +#define BLOCK ((void *)(GOLDEN_ARENA + 0x100)) /* heap block, 16 bytes */ +#define WILD ((void *)(GOLDEN_ARENA + 0x800)) /* never allocated */ + +void __real_free(void *ptr); +void __wrap_free(void *ptr) { + if ((uintptr_t)ptr - GOLDEN_ARENA < GOLDEN_ARENA_SIZE) { + return; + } + __real_free(ptr); +} + +static void golden_arena(void) { + void *arena = mmap((void *)GOLDEN_ARENA, GOLDEN_ARENA_SIZE, + PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED_NOREPLACE, -1, 0); + if (arena != (void *)GOLDEN_ARENA) { + fprintf(stderr, "golden: cannot map the arena\n"); + exit(2); + } +} + +/* Basic blocks (one repeated line) and one nondet value of each kind the + * Rust library logs. */ +static void golden_tracking(void) { + map2check_track_bb(3, "main"); + map2check_track_bb(4, "main"); + map2check_track_bb(3, "main"); + map2check_nondet_int(5, 0, 42, "main"); + map2check_nondet_uint(6, 0, (int)4000000000u, "main"); + map2check_nondet_char(7, 0, -3, "main"); + map2check_nondet_long(8, 0, -7, "main"); + map2check_nondet_double(9, 0, 2.5, "main"); +} + +/* int *p (stack slot of 8 bytes); &main; p = malloc(16). */ +static void golden_memory(void) { + map2check_alloca("p", SLOT_P, 8, 8, 10, 1); + map2check_function("main", FN_MAIN); + map2check_malloc(BLOCK, 16); + *(void **)SLOT_P = BLOCK; + map2check_add_store_pointer(SLOT_P, BLOCK, 1, "p", 11, "main"); +} + +#endif From e16cb5b8e1137d9a304242d84771a476f27b70f8 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 22:13:24 -0400 Subject: [PATCH 24/63] feat: write legacy property, log and witness files consumed by the Map2Check frontend (#19, #14) With an analysis mode set, the library now ends a run as the C runtime of Map2Check develop does: the first violation writes the property files and aborts, and map2check_success() writes NONE, runs the end-of-program verdict and returns. map2check_enable_witness() adds the witness files (track_bb_log.st, heap/allocation/list logs, klee_log.csv and map2check_checked_error). map2check_init() removes stale property files. Without a mode nothing is written and nothing aborts. Golden tests compare every file byte for byte with tests/legacy_golden. The JSON report stays as an additional channel, pinned by a snapshot. New semantic deltas: SD-16 (map2check_property__unknown kept on purpose) and SD-17 (logs rebuilt from this library's records). --- Cargo.lock | 1 + Cargo.toml | 1 + docs/api-deltas.md | 60 ++- docs/integration-guide.md | 20 +- docs/semantic-deltas.md | 100 +++- rust/src/ffi.rs | 282 ++++++++---- rust/src/ffi_golden_tests.rs | 424 +++++++++++++++++ rust/src/ffi_tests.rs | 44 ++ rust/src/memtrack/mod.rs | 34 ++ rust/src/output/json_report.snap | 44 ++ rust/src/output/legacy_files.rs | 752 +++++++++++++++++++++++++++++++ rust/src/output/mod.rs | 25 + rust/src/output/termination.rs | 176 ++++++++ rust/src/state.rs | 6 + tests/compat_docs.rs | 1 + tests/ffi_harness.c | 177 +++++++- 16 files changed, 2008 insertions(+), 139 deletions(-) create mode 100644 rust/src/ffi_golden_tests.rs create mode 100644 rust/src/output/json_report.snap create mode 100644 rust/src/output/legacy_files.rs create mode 100644 rust/src/output/termination.rs diff --git a/Cargo.lock b/Cargo.lock index bb22958..4cea788 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -186,6 +186,7 @@ dependencies = [ "mockall", "serde", "serde_json", + "tempfile", "thiserror", ] diff --git a/Cargo.toml b/Cargo.toml index 86194ab..218007a 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -19,6 +19,7 @@ thiserror = "2" [dev-dependencies] mockall = "0.13" +tempfile = "3" [features] libfuzzer = [] diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 1fd558e..7eead9a 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -43,19 +43,20 @@ build with `--features libfuzzer`. | Symbol | Rust declaration (`include/map2check.h`) | Legacy C library | Status | Notes | |---|---|---|---|---| -| `map2check_init` | `void map2check_init(void)` | `map2check_init` | same | Starts a fresh analysis through the `analysis_init` of the selected mode and keeps the options and the mode (#10, SD-14). Map2Check passes call `map2check_init(int)`; see the pass matrix. | +| `map2check_init` | `void map2check_init(void)` | `map2check_init` | same | Starts a fresh analysis through the `analysis_init` of the selected mode and keeps the options and the mode (#10, SD-14). With a mode set, also removes the `map2check_property*` files left in the current directory (see “Result files”). Map2Check passes call `map2check_init(int)`; see the pass matrix. | | `map2check_reset` | `void map2check_reset(void)` | — | new | Clears containers, result and step counter (#9). Keeps the options set by `map2check_set_memcleanup()` and `map2check_set_null_is_valid()` and the mode set by `map2check_set_mode()`: they are configuration, not analysis state (#10, SD-14). | -| `map2check_success` | `void map2check_success(void)` | `map2check_success` | same | Prints the JSON report (SD-01). | +| `map2check_success` | `void map2check_success(void)` | `map2check_success` | same | Prints the JSON report (SD-01). With a mode set it is develop's `map2check_success`: writes `NONE`, runs the end-of-program verdict, writes the witness files when enabled, and returns; a violating verdict writes its property and aborts (see “Result files”). | | `set_false_result` | `void set_false_result(int32_t prp, int32_t line_number, const char *function_name)` | `set_false_result` | same | `ViolatedProperty` keeps the C variant order (`caller_lib_result.h:22-31`); the appended `Assert` (8) and `Unknown` (9) are accepted; unknown values are ignored (SD-09). Ignored when a violation is already recorded (SD-03). | | `get_current_property` | `enum ViolatedProperty get_current_property(void)` | `get_current_property` | same | | | `get_current_step` | `uint64_t get_current_step(void)` | `long get_current_step(void)` | changed | Returns `uint64_t` instead of `long`. | | `get_next_step` | `uint64_t get_next_step(void)` | `get_next_step()` / `next_current_step()` | same | Advances and returns the step counter. `next_current_step` is not exported; see its row. | -| `print_all_containers_as_json` | `void print_all_containers_as_json(void)` | `print_all_containers_as_json` | changed | Report shape and whitespace differ (SD-01, SD-02; schema decision in #14). | +| `print_all_containers_as_json` | `void print_all_containers_as_json(void)` | `print_all_containers_as_json` | changed | Report shape and whitespace differ (SD-01, SD-02). An additional channel with its own schema, pinned by a snapshot test; Map2Check reads the result files instead (#14, see “Result files”). | | `vcc_reset_meta_data` | `void vcc_reset_meta_data(void)` | `vcc_reset_meta_data` | same | Host/test-only: the Map2Check passes never call it. It clears a recorded violation by design, bypassing the first-violation rule (SD-03). | | `map2check_set_memcleanup` | `void map2check_set_memcleanup(void)` | `map2check_set_memcleanup` | same | Selects `MEMSAFETY_MEMCLEANUP` instead of `MEMSAFETY_MEMTRACK` for the end-of-program leak (SD-04): an alias of the memcleanup mode for the leak tag, in every mode that checks leaks (SD-14). Kept by `map2check_init()` and `map2check_reset()`. | | `map2check_set_null_is_valid` | `void map2check_set_null_is_valid(void)` | `map2check_set_null_is_valid` | same | Makes a NULL load safe (SD-05). Kept by `map2check_init()` and `map2check_reset()`. | +| `map2check_enable_witness` | `void map2check_enable_witness(void)` | — | new | Turns on witness generation, the run-time counterpart of develop linking `WitnessGeneration.c` instead of `WitnessGenerationNone.c` (`caller.cpp:624-640`). A Map2Check run (mode set) then also writes `track_bb_log.st`, the memory logs and, on a violation, `klee_log.csv` and `map2check_checked_error`. Kept by `map2check_init()` and `map2check_reset()`. The frontend links a constructor that calls it into witness binaries. See “Result files”. | | `map2check_set_mode` | `void map2check_set_mode(int32_t mode)` | — | new | Selects the analysis mode at run time; `mode` is one of the `enum AnalysisMode` values the header declares (`AnalysisMode_None` 0, `_Memtrack` 1, `_Memcleanup` 2, `_Overflow` 3, `_Assert` 4, `_Target` 5). Develop selects it at link time. Called before `map2check_init()`; kept by `map2check_init()` and `map2check_reset()`. Never called: mode unset, every check active. Any other value is ignored with a diagnostic on stderr and leaves the mode unchanged. See “Analysis modes” (SD-14). | -| `map2check_reach_target` | `void map2check_reach_target(const char *func, int32_t scope, int32_t line)` | — | new | Replaces the pass symbol `map2check_target_function`. Records `REACHABILITY` (develop `TARGET-REACHED`) at `line` in `func`, in every mode; `scope` is ignored. Does not end the process (SD-15). | +| `map2check_reach_target` | `void map2check_reach_target(const char *func, int32_t scope, int32_t line)` | — | new | Replaces the pass symbol `map2check_target_function`. Records `REACHABILITY` (develop `TARGET-REACHED`) at `line` in `func`, in every mode; `scope` is ignored. With a mode set, ends the run like any first violation; unset, it returns (SD-15). | | `map2check_save_nondet_log_int` | `non_det_log_t *map2check_save_nondet_log_int(int line, int scope, enum var_type_t type_var, int *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_uint` | `non_det_log_t *map2check_save_nondet_log_uint(int line, int scope, enum var_type_t type_var, unsigned int *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_long` | `non_det_log_t *map2check_save_nondet_log_long(int line, int scope, enum var_type_t type_var, long *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | @@ -137,7 +138,7 @@ Map2Check. | `map2check_load` | `(ptr, i64)` | pending | Core `map2check_check_load` takes 5 arguments (#21). | | `map2check_add_store_pointer` | 6 arguments | pending | Core `map2check_map_store_pointer` differs in order and types (#21). | | `map2check_function` | `(ptr, ptr)` | pending | Core equivalent: `map2check_map_funct_address` (#21). | -| `map2check_target_function` | `(ptr, i32, i32)` | pending | Core equivalent: `map2check_reach_target`, same arguments; records and returns instead of ending the run (SD-15). The pass switches to it later in the cutover. | +| `map2check_target_function` | `(ptr, i32, i32)` | pending | Core equivalent: `map2check_reach_target`, same arguments; with a mode set it ends the run as develop does, unset it records and returns (SD-15). The pass switches to it later in the cutover. | | `map2check_assume` | — | pending | #21. | | `map2check_assume_loop` | — | pending | #21. | | `map2check_crab_assume` | — | pending | #21. | @@ -153,10 +154,9 @@ Map2Check. | `__VERIFIER_assert` | `(int)` | same | Feature `libfuzzer` (SD-08). | | `__VERIFIER_error` | `(void)` | same | Feature `libfuzzer` (SD-08). | -The passes also expect the result channel of the embedded library: files -named `map2check_property_*` that the Map2Check frontend parses. This -library only prints the JSON report on stdout (SD-01). The file channel is -tracked in #19. +The passes also expect the result channel of the embedded library: the +files that the Map2Check frontend parses. This library writes them when a +mode is set (#19); see “Result files”. ## Analysis modes @@ -181,6 +181,48 @@ verdict; it does not apply the memcleanup-mode reachability rule. `ViolatedProperty` gained `Assert` (8) and `Unknown` (9), appended after `None`; `Reachability` is the target-reached property. +## Result files + +The Map2Check frontend does not read JSON. It reads the files that the +runtime writes into the current directory and learns of a violation from +the process aborting (`modules/frontend/utils/tools.cpp`, `caller.cpp`, +`counter_example/`). That file channel is the result channel Map2Check +consumes. The JSON report of `print_all_containers_as_json()` and +`map2check_success()` is an additional channel with its own schema +(SD-01), pinned by `rust/src/output/mod.rs::json_report_snapshot` (#14). + +The channel is tied to the mode. When `map2check_set_mode()` was called +(a Map2Check-instrumented program): + +- `map2check_init()` removes stale `map2check_property*` files; +- the first violation writes the property files, then, with witness on, + the witness files, runs the mode's `analysis_destroy`, writes + `map2check_checked_error` (witness on) and calls `abort()`, as develop's + `map2check_error` → `map2check_exit` (`Map2CheckFunctions.c:72-102`); +- `map2check_success()` writes `NONE`, runs the end-of-program verdict + and, if it holds, writes the witness files (witness on) and returns; + otherwise it ends the run as a violation. + +When the mode is unset (direct API users, `tests/ffi_harness.c`) nothing +is written and nothing ends the process: violations are only recorded. +In memtrack and memcleanup modes `map2check_map_malloc` and +`map2check_map_calloc` with a NULL block report `UNKNOWN`, as develop's +`map2check_malloc` does. + +| File | Develop writer | Rust writer (`rust/src/output/`) | When | +|---|---|---|---| +| `map2check_property` | `PropertyGenerator.c:50-106` | `legacy_files::write_property_files` | mode set: on the first violation (every property but `UNKNOWN`) and `NONE` on `map2check_success` | +| `map2check_property_{free,deref,overflow,assert,memtrack,memcleanup}` | `PropertyGenerator.c:24-48`, `:108-130` | `legacy_files::write_property_files` | mode set: with `map2check_property` for that property | +| `map2check_property__unknown` | `PropertyGenerator.c:16`, `:24-28` | `legacy_files::write_property_files` | mode set: `UNKNOWN` (double underscore kept, SD-16) | +| `track_bb_log.st` | `TrackBBLog.c:35-44` | `legacy_files::write_trackbb_log` | mode set, witness on: end of run | +| `heap_log.csv`, `allocation_log.csv`, `list_log.csv` | `HeapLog.c:54-66`, `AllocationLog.c:161-174`, `ListLog.c:138-157` | `legacy_files::LegacyLogs` | memtrack or memcleanup mode, witness on: end of run (SD-17) | +| `klee_log.csv` | `NonDetLog.c:17-60` (via `WitnessGeneration.c:22-24`) | `legacy_files::LegacyLogs::write_nondet_log` | mode set, witness on: end of a violating run | +| `map2check_checked_error` | `WitnessGeneration.c:28-32` | `legacy_files::write_checked_error` | mode set, witness on: before the abort | + +`tests/legacy_golden/` holds the files the develop C runtime writes for one +scenario per verdict, and `rust/src/ffi_golden_tests.rs` checks that this +library writes the same bytes for the same calls. + ## Discontinued: `map2check_gen_data_fuzzer_pchar` / `_loff_t` / `_sector_t` **Status:** discontinued, not ported to `feature/rust`. diff --git a/docs/integration-guide.md b/docs/integration-guide.md index 4ed0090..3c2481f 100644 --- a/docs/integration-guide.md +++ b/docs/integration-guide.md @@ -183,8 +183,22 @@ print: `property` takes one of `OVERFLOW`, `MEMSAFETY_FREE`, `MEMSAFETY_DEREF`, `MEMSAFETY_MEMTRACK`, `MEMSAFETY_MEMCLEANUP`, `REACHABILITY`, `CONCURRENCY`, `ASSERT` and `UNKNOWN`. Nondet `type` takes one of `INT`, `UINT`, `LONG`, `CHAR`, -`FLOAT` and `DOUBLE`. The schema is not final: container and field names -are still being reconciled with the existing consumer in #14 (SD-01). +`FLOAT` and `DOUBLE`. Map2Check does not read this JSON: it reads the +result files described below. The JSON is an additional channel with its +own schema, pinned by a snapshot test (#14, SD-01). + +## Result files for Map2Check + +A Map2Check-instrumented program calls `map2check_set_mode()` before +`map2check_init()`, and the witness binary also calls +`map2check_enable_witness()`. With a mode set, the library writes into the +current directory the files the Map2Check frontend reads, byte for byte as +the C runtime of Map2Check `develop` does: `map2check_property*`, and with +witness on `track_bb_log.st`, `heap_log.csv`, `allocation_log.csv`, +`list_log.csv`, `klee_log.csv` and `map2check_checked_error`. The first +violation ends the process with `abort()`; `map2check_success()` returns +when the end-of-program verdict holds. Without a mode nothing is written +and nothing aborts. See “Result files” in `api-deltas.md`. ## Known gaps @@ -192,8 +206,6 @@ are still being reconciled with the existing consumer in #14 (SD-01). different symbol family (`map2check_track_bb`, `map2check_alloca`, …) and cannot link against this library yet. See the pass matrix in `api-deltas.md` and #21. -- **Result files:** the Map2Check frontend reads `map2check_property_*` - files; this library only prints JSON (#19). - **KLEE:** no KLEE build exists yet; the decision is #20. - **Thread safety:** the state sits behind one global mutex. The calls are safe from any thread but serialise, and a callback must not re-enter the diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 06abef3..249f0ca 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -30,11 +30,12 @@ entry and its tests in the same change. | `suspected regression` | Probably wrong; an issue tracks adjudication. | | `pending decision` | Waits on the cited issue. | | `fixed` | Was a regression; the Rust library now matches the C runtime of Map2Check `develop`. The entry cites the fixing issue and the `develop` source. | +| `C behaviour preserved on purpose` | The C runtime of Map2Check `develop` does something odd and the Rust library does the same, because a consumer depends on the exact output. | ### SD-01 — JSON report shape -**Classification:** pending decision (#14) -**Tests:** `tests/compat_docs.rs::integration_guide_json_examples_match_the_serializer`, `rust/src/output/mod.rs::output_is_valid_json` +**Classification:** intentional (#14) +**Tests:** `tests/compat_docs.rs::integration_guide_json_examples_match_the_serializer`, `rust/src/output/mod.rs::output_is_valid_json`, `rust/src/output/mod.rs::json_report_snapshot` **Rust:** `print_all_containers_as_json()`, `map2check_success()` and a failing `map2check_is_valid_assert()` print one pretty-printed JSON object @@ -51,6 +52,12 @@ lists container names `container_trackbb_log`, `container_nondet_log` and `container_memtrack_log`, and asks whether a top-level `function_name` is part of the schema. Whitespace alone is not a semantic difference. +**Decision (#14):** Map2Check does not read this JSON. It reads the result +files that the library writes when an analysis mode is set (#19, see +“Result files” in [`api-deltas.md`](api-deltas.md#result-files)). The JSON +is an additional channel with its own schema, kept as it is and pinned by +`json_report_snapshot`. + **Reproduce:** ```c @@ -88,9 +95,10 @@ print_all_containers_as_json(); /* Container_AllocationLog present */ **Classification:** fixed (#36) **Tests:** `rust/src/ffi_tests.rs::sd03_safe_check_keeps_earlier_violation`, `rust/src/ffi_tests.rs::sd03_later_violation_does_not_overwrite_first` -**Rust:** a check that finds a violation records it and returns. The process -keeps running (ending it, with the result files, is Task 2.4 of the cutover -plan). The first recorded violation is final: `AnalysisState::set_false` +**Rust:** with an analysis mode set (a Map2Check run), the first violation +writes the result files and aborts, as in develop (Task 2.4, #19). With the +mode unset, a check that finds a violation records it and returns, and the +process keeps running. Either way the first recorded violation is final: `AnalysisState::set_false` ignores every later violation, whether it comes from a memory check, an overflow check, an assert or `set_false_result`, and the memory checks (`map2check_check_load`, `map2check_check_free`, @@ -456,10 +464,13 @@ map2check_check_mem_endprog(); /* MemsafetyMemcleanup */ **Tests:** `rust/src/ffi_tests.rs::map2check_reach_target_records_and_returns`, `rust/src/ffi_tests.rs::reach_target_keeps_the_first_violation`, `rust/src/ffi_tests.rs::reach_target_works_in_every_mode_and_accepts_null_name` **Rust:** `map2check_reach_target(func, scope, line)` records -`REACHABILITY` at `line` in `func` and returns. The first violation stays -final (SD-03). It works in every mode and ignores `scope`. Ending the run -with the result files belongs to Task 2.4 of the cutover plan; until then -the program keeps running after the target. +`REACHABILITY` at `line` in `func`. The first violation stays final +(SD-03). It works in every mode and ignores `scope`. With a mode set it +ends the run like develop: `map2check_property` gets `TARGET-REACHED`, the +witness files are written when enabled, and the process aborts (Task 2.4, +`rust/src/ffi_golden_tests.rs::target_reached_matches_c_runtime`). The +limitation that remains is the unset mode, where the program keeps running +after the target. **C runtime of Map2Check `develop`:** `map2check_target_function` writes `TARGET-REACHED` and calls `map2check_error()`, which ends the process @@ -473,3 +484,74 @@ map2check_set_mode(AnalysisMode_Target); map2check_init(); map2check_reach_target("reach_error", 0, 12); /* Reachability, line 12; returns */ ``` + +### SD-16 — UNKNOWN goes to `map2check_property__unknown` + +**Classification:** C behaviour preserved on purpose +**Tests:** `rust/src/output/legacy_files.rs::unknown_uses_legacy_double_underscore_name`, `rust/src/ffi_golden_tests.rs::unknown_matches_c_runtime` + +**Rust:** an `UNKNOWN` verdict (`set_false_result(9, ...)`, or a NULL block +from `map2check_map_malloc`/`_calloc` in the memtrack and memcleanup modes) +writes `UNKNOWN\n` to `map2check_property__unknown`, with two +underscores, and writes no `map2check_property`. + +**C runtime of Map2Check `develop`:** `PropertyGenerator.c:16` names the +file `map2check_property__unknown`, and `write_property_unknown` (`:24-28`) +is the only UNKNOWN writer that runs (`AnalysisModeMemtrack.c:305-311`). +The frontend looks for `map2check_property_unknown`, with one underscore +(`modules/frontend/utils/tools.cpp:114`), so it never finds the file and +reports UNKNOWN because `map2check_property` is missing (`:98-103`). The +verdict is the same either way; the name is kept so the bytes on disk match +develop. + +**Reproduce:** + +```c +map2check_set_mode(AnalysisMode_Memtrack); +map2check_init(); +map2check_map_malloc(NULL, 4); /* writes map2check_property__unknown, aborts */ +``` + +### SD-17 — The memory and nondet logs are rebuilt from this library's records + +**Classification:** accepted limitation +**Tests:** `rust/src/ffi_golden_tests.rs::false_free_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::none_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::false_memtrack_matches_c_runtime`, `rust/src/output/legacy_files.rs::nondet_rows_use_develop_types_and_value_formats` + +**Rust:** `heap_log.csv`, `allocation_log.csv`, `list_log.csv` and +`klee_log.csv` are rebuilt at the end of the run by replaying the memory +and nondet records in order with the rules of `AnalysisModeMemtrack.c`, +including develop's own step counter. For the calls of +`tests/legacy_golden/drivers/` the bytes match develop. These cases +differ: + +- **Pointer variable of a free.** `map2check_map_free` receives the block, + not the pointer variable, so the FREE row of `list_log.csv` names the + variable of the newest row that points to the block. Develop names the + variable passed to `map2check_free`. +- **Failing `free`.** Develop's `map2check_free` appends that FREE row + before it checks (`AnalysisModeMemtrack.c:346-358`). Here + `map2check_check_free` aborts before `map2check_map_free` records it, so + the row is missing from `list_log.csv`. +- **`klee_log.csv`.** Written on every violating witness run. Develop + writes it only with the AFL++ or KLEE generator; `NonDetGeneratorNone.c` + writes none (`:42`). A `float` value, which develop has no logger for, is + logged as a double (type 15). +- **Long function names.** Develop copies names into `char[35]` with + `strncpy` (`FUNCTION_MAX_LENGTH_NAME`), which leaves names of 35 or more + characters unterminated; this library writes the whole name. +- **NULL names.** Develop prints `(null)` for a NULL name; this library + prints an empty string, as everywhere else (SD-09). + +**Reproduce:** + +```c +map2check_set_mode(AnalysisMode_Memtrack); +map2check_enable_witness(); +map2check_init(); +map2check_map_malloc((void *)0x3000, 8); +map2check_map_store_pointer((void *)0x10, (void *)0x3000, 0, "p", 1, "main"); +map2check_map_store_pointer((void *)0x20, (void *)0x3000, 0, "q", 2, "main"); +map2check_check_free("p", (void *)0x3000, 0, 3, "main"); +map2check_map_free("p", (void *)0x3000, 0, 3, "main"); +/* list_log.csv: the FREE row names 0x20 (q); develop names 0x10 (p) */ +``` diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index 90842de..e0d81c0 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -32,10 +32,12 @@ use crate::analysismode::VccContext; use crate::bbtrack::BasicBlockEntry; use crate::caller::ViolatedProperty; use crate::error::Map2CheckError; +use crate::memtrack::MemOrigin; use crate::memtrack::MemTrackEntry; use crate::nondet::NonDetEntry; use crate::nondet::NonDetValue; use crate::output; +use crate::output::termination; use crate::state; // — Helpers ————————————————————————————————————————————————————————————————— @@ -58,17 +60,29 @@ fn log_error(e: &Map2CheckError) { eprintln!("[map2check] error: {e}"); } +/// Records a violation (the first one is final, SD-03). When it is the +/// first and a Map2Check analysis mode is set, the run ends here as in +/// develop's `map2check_error()`: the result files are written and the +/// process aborts (`output::termination`). `end_of_program` marks the +/// end-of-program verdict. +fn record_violation(property: ViolatedProperty, line: u32, fname: &str, end_of_program: bool) { + match state::with_state(|s| { + let first = s.result.ok; + s.set_false(property, line, fname); + Ok(first) + }) { + Ok(true) => termination::on_violation(end_of_program), + Ok(false) => {} + Err(e) => log_error(&e), + } +} + /// Runs `checker`, records a violation if it finds one, and returns whether /// the check passed (`false` on a violation or an internal error). fn run_vcc_check(checker: impl VccChecker, ctx: &VccContext, line: u32, fname: &str) -> bool { match checker.check(ctx) { Ok(crate::analysismode::VccOutcome::Violated { property }) => { - if let Err(e) = state::with_state(|s| { - s.set_false(property, line, fname); - Ok(()) - }) { - log_error(&e); - } + record_violation(property, line, fname, false); false } Ok(crate::analysismode::VccOutcome::Safe) => true, @@ -127,9 +141,18 @@ pub(crate) fn ffi_guard(f: impl FnOnce() -> T, default: T) -> T { // ── Initialisation ──────────────────────────────────────────────────────────── /// Initialises all containers. Must be called before any tracking function. +/// With an analysis mode set, also removes the property files +/// (`map2check_property*`) that an earlier run left in the current +/// directory, so they cannot be read as this run's verdict. #[no_mangle] pub extern "C" fn map2check_init() { - ffi_guard(state::init, ()); + ffi_guard( + || { + state::init(); + termination::on_init(); + }, + (), + ); } /// Resets the global analysis state without aborting the caller. @@ -140,13 +163,26 @@ pub extern "C" fn map2check_reset() { // ── Result management ───────────────────────────────────────────────────────── -/// Records a verification success and prints the JSON report. +/// End of the analysed program. Prints the JSON report. With an analysis +/// mode set it is develop's `map2check_success()` +/// (`Map2CheckFunctions.c:67-70`, `:89-102`): writes `NONE` to +/// `map2check_property`, runs the end-of-program verdict of the mode, writes +/// the witness files when `map2check_enable_witness()` was called, and +/// returns; when the verdict is a violation it writes that property instead +/// and aborts. The JSON is printed after the verdict. #[no_mangle] pub extern "C" fn map2check_success() { ffi_guard( || { - if let Err(e) = state::with_state(|s| output::print_json(s)) { - log_error(&e); + let print = |s: &state::AnalysisState| { + if let Err(e) = output::print_json(s) { + log_error(&e); + } + }; + if !termination::on_success(print) { + if let Err(e) = state::with_state(|s| output::print_json(s)) { + log_error(&e); + } } }, (), @@ -171,12 +207,7 @@ pub unsafe extern "C" fn set_false_result( }; // SAFETY: caller guarantees function_name is a valid C string or null. let fname = unsafe { c_str(function_name) }; - if let Err(e) = state::with_state(|s| { - s.set_false(property, line_number as u32, fname); - Ok(()) - }) { - log_error(&e); - } + record_violation(property, line_number as u32, fname, false); }, (), ); @@ -270,6 +301,24 @@ pub extern "C" fn map2check_set_null_is_valid() { ); } +/// Turns on witness generation, as linking `WitnessGeneration.c` instead of +/// `WitnessGenerationNone.c` does in develop (`caller.cpp:624-640`). Effect: +/// when a Map2Check run ends (an analysis mode is set), the library also +/// writes `track_bb_log.st`, `heap_log.csv`, `allocation_log.csv` and +/// `list_log.csv` (memory modes), and on a violation `klee_log.csv` and +/// `map2check_checked_error`. Configuration: kept by `map2check_init()` and +/// `map2check_reset()`. The Map2Check frontend links a constructor that +/// calls it into the witness binary. +#[no_mangle] +pub extern "C" fn map2check_enable_witness() { + ffi_guard( + || { + state::update_config(|c| c.witness = true); + }, + (), + ); +} + /// Selects the analysis mode, as the frontend of Map2Check `develop` does by /// linking one `AnalysisMode*.c` per run (`caller.cpp:575-612`). The /// instrumented program calls it before `map2check_init()` with one of the @@ -319,12 +368,7 @@ pub(crate) unsafe fn mark_target_reached( ) { // SAFETY: caller guarantees func is a valid C string or null. let fname = unsafe { c_str(func) }; - if let Err(e) = state::with_state(|s| { - s.set_false(ViolatedProperty::Reachability, line as u32, fname); - Ok(()) - }) { - log_error(&e); - } + record_violation(ViolatedProperty::Reachability, line as u32, fname, false); } /// Called by the target pass when the target function (e.g. `reach_error`) @@ -561,20 +605,23 @@ pub unsafe extern "C" fn map2check_map_alloca( let addr = ptr_address as usize; if let Err(e) = state::with_state(|s| { let step = s.next_step(); - s.memtrack.push(MemTrackEntry::new( - step, - line_number as u32, - scope as u32, - addr, - addr, - false, - false, - name, - "", - size as usize, - size_primitive as usize, - false, - )); + s.memtrack.push( + MemTrackEntry::new( + step, + line_number as u32, + scope as u32, + addr, + addr, + false, + false, + name, + "", + size as usize, + size_primitive as usize, + false, + ) + .with_origin(MemOrigin::Alloca), + ); Ok(()) }) { log_error(&e); @@ -616,20 +663,23 @@ pub unsafe extern "C" fn map2check_map_non_static_alloca( let addr = ptr_address as usize; if let Err(e) = state::with_state(|s| { let step = s.next_step(); - s.memtrack.push(MemTrackEntry::new( - step, - line_number as u32, - scope as u32, - addr, - addr, - false, - false, - name, - "", - bytes, - size_primitive as usize, - false, - )); + s.memtrack.push( + MemTrackEntry::new( + step, + line_number as u32, + scope as u32, + addr, + addr, + false, + false, + name, + "", + bytes, + size_primitive as usize, + false, + ) + .with_origin(MemOrigin::Alloca), + ); Ok(()) }) { log_error(&e); @@ -659,20 +709,23 @@ pub unsafe extern "C" fn map2check_map_funct_address( let pointer_width = std::mem::size_of::<*const std::os::raw::c_void>(); if let Err(e) = state::with_state(|s| { let step = s.next_step(); - s.memtrack.push(MemTrackEntry::new( - step, - 0, - 0, - addr, - addr, - false, - false, - name, - "", - pointer_width, - pointer_width, - false, - )); + s.memtrack.push( + MemTrackEntry::new( + step, + 0, + 0, + addr, + addr, + false, + false, + name, + "", + pointer_width, + pointer_width, + false, + ) + .with_origin(MemOrigin::Function), + ); Ok(()) }) { log_error(&e); @@ -682,6 +735,26 @@ pub unsafe extern "C" fn map2check_map_funct_address( ); } +/// In the memory modes, a NULL block is develop's UNKNOWN: `map2check_malloc` +/// writes `map2check_property__unknown` and ends the run before recording +/// anything (`AnalysisModeMemtrack.c:305-311`, `AnalysisModeMemcleanup.c:295-300`). +/// Returns whether the call was handled that way. +fn malloc_returned_null(ptr_address: *const std::os::raw::c_void) -> bool { + let memory_mode = matches!( + state::config().mode, + Some(AnalysisMode::Memtrack | AnalysisMode::Memcleanup) + ); + if !(memory_mode && ptr_address.is_null()) { + return false; + } + record_violation(ViolatedProperty::Unknown, 0, "", false); + true +} + +/// Records a heap allocation of `size` bytes at `ptr_address`. In the +/// memtrack and memcleanup modes a NULL `ptr_address` reports `UNKNOWN` +/// instead, as develop does. +/// /// # Safety /// `ptr_address` may be null (null malloc result is valid C). #[no_mangle] @@ -691,6 +764,9 @@ pub unsafe extern "C" fn map2check_map_malloc(ptr_address: *const std::os::raw:: if size < 0 { return; } + if malloc_returned_null(ptr_address) { + return; + } let addr = ptr_address as usize; if let Err(e) = state::with_state(|s| { let step = s.next_step(); @@ -709,7 +785,7 @@ pub unsafe extern "C" fn map2check_map_malloc(ptr_address: *const std::os::raw:: false, ); entry.set_malloc(); - s.memtrack.push(entry); + s.memtrack.push(entry.with_origin(MemOrigin::Malloc)); Ok(()) }) { log_error(&e); @@ -719,6 +795,10 @@ pub unsafe extern "C" fn map2check_map_malloc(ptr_address: *const std::os::raw:: ); } +/// Records a heap allocation of `quantity * size` bytes at `ptr_address`. +/// In the memtrack and memcleanup modes a NULL `ptr_address` reports +/// `UNKNOWN` instead, as develop's `map2check_calloc` does. +/// /// # Safety /// `ptr_address` may be null. #[no_mangle] @@ -732,6 +812,9 @@ pub unsafe extern "C" fn map2check_map_calloc( if quantity < 0 || size < 0 { return; } + if malloc_returned_null(ptr_address) { + return; + } let quantity = quantity as usize; let size = size as usize; if size.checked_mul(quantity).is_none() { @@ -746,7 +829,7 @@ pub unsafe extern "C" fn map2check_map_calloc( if !entry.set_calloc(quantity) { return Ok(()); } - s.memtrack.push(entry); + s.memtrack.push(entry.with_origin(MemOrigin::Malloc)); Ok(()) }) { log_error(&e); @@ -789,7 +872,7 @@ pub unsafe extern "C" fn map2check_map_free( false, ); entry.set_free(); - s.memtrack.push(entry); + s.memtrack.push(entry.with_origin(MemOrigin::Free)); Ok(()) }) { log_error(&e); @@ -819,20 +902,23 @@ pub unsafe extern "C" fn map2check_map_store_pointer( let points_to = value as usize; if let Err(e) = state::with_state(|s| { let step = s.next_step(); - s.memtrack.push(MemTrackEntry::new( - step, - line_number as u32, - scope, - var_addr, - points_to, - false, - false, - name, - fname, - 0, - 1, - false, - )); + s.memtrack.push( + MemTrackEntry::new( + step, + line_number as u32, + scope, + var_addr, + points_to, + false, + false, + name, + fname, + 0, + 1, + false, + ) + .with_origin(MemOrigin::StorePointer), + ); Ok(()) }) { log_error(&e); @@ -865,11 +951,15 @@ pub unsafe extern "C" fn map2check_is_valid_assert( let checker = AssertChecker { expression }; match checker.check(&ctx) { Ok(crate::analysismode::VccOutcome::Violated { property }) => { - if let Err(e) = state::with_state(|s| { + match state::with_state(|s| { + let first = s.result.ok; s.set_false(property, line_number as u32, fname); - output::print_json(s) + output::print_json(s)?; + Ok(first) }) { - log_error(&e); + Ok(true) => termination::on_violation(false), + Ok(false) => {} + Err(e) => log_error(&e), } } Ok(crate::analysismode::VccOutcome::Safe) => {} @@ -1012,12 +1102,7 @@ pub unsafe extern "C" fn map2check_check_load( // SAFETY: caller guarantees function_name is valid or null. let fname = unsafe { c_str(function_name) }; if size < 0 { - if let Err(e) = state::with_state(|s| { - s.set_false(ViolatedProperty::MemsafetyDeref, line as u32, fname); - Ok(()) - }) { - log_error(&e); - } + record_violation(ViolatedProperty::MemsafetyDeref, line as u32, fname, false); return; } let ctx = VccContext::new(line as u32, scope, fname); @@ -1120,7 +1205,7 @@ pub unsafe extern "C" fn map2check_check_free_resolved_address( step, line, 0, addr, addr, true, false, "", fname, 0, 1, false, ); entry.set_free(); - s.memtrack.push(entry); + s.memtrack.push(entry.with_origin(MemOrigin::FreeResolved)); Ok(()) }) { log_error(&e); @@ -1173,13 +1258,10 @@ pub extern "C" fn map2check_check_mem_endprog() { ffi_guard( || { let config = state::config(); - if let Err(e) = state::with_state(|s| { - if let Some(property) = mode::analysis_is_program_correct(config, s) { - s.set_false(property, 0, "end_of_program"); - } - Ok(()) - }) { - log_error(&e); + match state::with_state(|s| Ok(mode::analysis_is_program_correct(config, s))) { + Ok(Some(property)) => record_violation(property, 0, "end_of_program", true), + Ok(None) => {} + Err(e) => log_error(&e), } }, (), @@ -1189,3 +1271,7 @@ pub extern "C" fn map2check_check_mem_endprog() { #[cfg(test)] #[path = "ffi_tests.rs"] mod tests; + +#[cfg(test)] +#[path = "ffi_golden_tests.rs"] +mod golden_tests; diff --git a/rust/src/ffi_golden_tests.rs b/rust/src/ffi_golden_tests.rs new file mode 100644 index 0000000..7988ae2 --- /dev/null +++ b/rust/src/ffi_golden_tests.rs @@ -0,0 +1,424 @@ +// SPDX-License-Identifier: GPL-3.0-only +// +// Golden tests of the file channel (#19): each scenario makes, through the +// exports of this library, the calls that a driver in +// `tests/legacy_golden/drivers/` makes on the C runtime of Map2Check +// `develop`, and every file it writes must equal, byte for byte, the file +// that runtime wrote in `tests/legacy_golden//`. +// +// The scenarios end the process (abort on a violation, as develop does), so +// each one runs in a child process: the parent re-runs this test binary +// filtered to one `*_child` test, with `MAP2CHECK_TEST_OUTPUT_DIR` pointing to +// a fresh temporary directory, and then compares the files. Without that +// variable a test build writes no files and never aborts +// (`output::termination::output_dir`). + +use std::collections::BTreeSet; +use std::os::raw::{c_char, c_long, c_void}; +use std::os::unix::process::ExitStatusExt; +use std::path::Path; + +use super::*; +use crate::analysismode::mode::AnalysisMode; + +const CHILD_ENV: &str = "MAP2CHECK_GOLDEN_CHILD"; +const STARTED: &str = "map2check-golden-child-started"; + +/// Fixed addresses of `tests/legacy_golden/drivers/golden.h`. +const ARENA: usize = 0x1000_0000; +const SLOT_P: usize = ARENA; +const FN_MAIN: usize = ARENA + 0x40; +const BLOCK: usize = ARENA + 0x100; +const WILD: usize = ARENA + 0x800; + +fn addr(a: usize) -> *const c_void { + a as *const c_void +} + +fn is_child() -> bool { + std::env::var_os(CHILD_ENV).is_some() +} + +/// Runs `ffi::golden_tests::` in a child process whose output +/// directory is `dir`. +fn run_child(name: &str, dir: &Path) -> std::process::Output { + let exe = std::env::current_exe().unwrap(); + let output = std::process::Command::new(exe) + .args(["--exact", &format!("ffi::golden_tests::{name}")]) + .args(["--nocapture", "--test-threads=1"]) + .env(CHILD_ENV, "1") + .env(crate::output::termination::OUTPUT_DIR_ENV, dir) + .output() + .unwrap(); + let stdout = String::from_utf8_lossy(&output.stdout); + assert!( + stdout.contains(STARTED), + "o processo filho não rodou `{name}`:\n{stdout}" + ); + output +} + +/// Names of the regular files in `dir`, without the `.exit_status` marker. +fn files_in(dir: &Path) -> BTreeSet { + std::fs::read_dir(dir) + .unwrap() + .map(|e| e.unwrap().file_name().into_string().unwrap()) + .filter(|name| name != ".exit_status") + .collect() +} + +/// Runs the `` scenario and compares its files and its end with +/// `tests/legacy_golden//`. +fn assert_matches_golden(tag: &str, child: &str) { + let golden = Path::new(env!("CARGO_MANIFEST_DIR")) + .join("tests/legacy_golden") + .join(tag); + let dir = tempfile::tempdir().unwrap(); + let output = run_child(child, dir.path()); + let stdout = String::from_utf8_lossy(&output.stdout); + let stderr = String::from_utf8_lossy(&output.stderr); + + let status = std::fs::read_to_string(golden.join(".exit_status")).unwrap(); + match status.trim() { + "134" => assert_eq!( + output.status.signal(), + Some(6), + "{tag}: esperava SIGABRT\n{stdout}\n{stderr}" + ), + "0" => assert_eq!( + output.status.code(), + Some(0), + "{tag}: esperava saída 0\n{stdout}\n{stderr}" + ), + other => panic!("{tag}: .exit_status inesperado `{other}`"), + } + + assert_eq!( + files_in(dir.path()), + files_in(&golden), + "{tag}: conjunto de arquivos" + ); + for name in files_in(&golden) { + let expected = std::fs::read(golden.join(&name)).unwrap(); + let actual = std::fs::read(dir.path().join(&name)).unwrap(); + assert_eq!( + String::from_utf8_lossy(&actual), + String::from_utf8_lossy(&expected), + "{tag}/{name}" + ); + assert_eq!(actual, expected, "{tag}/{name} (bytes)"); + } +} + +/// `golden_tracking()`: basic blocks (one repeated line) and one nondet value +/// of each kind. +fn tracking() { + let mut int_value: i32 = 42; + let mut uint_value: u32 = 4_000_000_000; + let mut char_value: c_char = -3; + let mut long_value: c_long = -7; + let mut double_value: f64 = 2.5; + // SAFETY: the values outlive the calls; the names are C-string literals. + unsafe { + map2check_save_basic_block_log(3, c"main".as_ptr()); + map2check_save_basic_block_log(4, c"main".as_ptr()); + map2check_save_basic_block_log(3, c"main".as_ptr()); + map2check_save_nondet_log_int(5, 0, 0, &mut int_value, c"main".as_ptr()); + map2check_save_nondet_log_uint(6, 0, 13, &mut uint_value, c"main".as_ptr()); + map2check_save_nondet_log_char(7, 0, 1, &mut char_value, c"main".as_ptr()); + map2check_save_nondet_log_long(8, 0, 4, &mut long_value, c"main".as_ptr()); + map2check_save_nondet_log_double(9, 0, 15, &mut double_value, c"main".as_ptr()); + } +} + +/// `golden_memory()`: `int *p` (8-byte slot), `&main`, `p = malloc(16)`. +fn memory() { + // SAFETY: fake addresses are never dereferenced; names are C strings. + unsafe { + map2check_map_alloca(c"p".as_ptr(), addr(SLOT_P), 8, 8, 10, 1); + map2check_map_funct_address(c"main".as_ptr(), addr(FN_MAIN)); + map2check_map_malloc(addr(BLOCK), 16); + map2check_map_store_pointer( + addr(SLOT_P), + addr(BLOCK), + 1, + c"p".as_ptr(), + 11, + c"main".as_ptr(), + ); + } +} + +/// `free(p)` as the pass emits it: the check, then the release. +fn free_p() { + // SAFETY: as in `memory`. + unsafe { + map2check_check_free(c"p".as_ptr(), addr(BLOCK), 1, 12, c"main".as_ptr()); + map2check_map_free(c"p".as_ptr(), addr(BLOCK), 1, 12, c"main".as_ptr()); + } +} + +/// Starts the analysis as the instrumented witness binary does. +fn start(mode: AnalysisMode) { + println!("{STARTED}"); + map2check_set_mode(mode as i32); + map2check_enable_witness(); + map2check_init(); + tracking(); +} + +#[test] +fn false_free_child() { + if is_child() { + start(AnalysisMode::Memtrack); + memory(); + free_p(); + // SAFETY: as in `memory`. + unsafe { map2check_check_free_resolved_address(addr(BLOCK), 13, c"main".as_ptr(), 0) }; + map2check_success(); + } +} + +#[test] +fn false_free_matches_c_runtime() { + assert_matches_golden("FALSE-FREE", "false_free_child"); +} + +#[test] +fn false_deref_child() { + if is_child() { + start(AnalysisMode::Memtrack); + memory(); + // SAFETY: as in `memory`. + unsafe { map2check_check_load(addr(WILD), 14, 0, 4, c"main".as_ptr()) }; + map2check_success(); + } +} + +#[test] +fn false_deref_matches_c_runtime() { + assert_matches_golden("FALSE-DEREF", "false_deref_child"); +} + +#[test] +fn false_memtrack_child() { + if is_child() { + start(AnalysisMode::Memtrack); + memory(); + map2check_success(); + } +} + +#[test] +fn false_memtrack_matches_c_runtime() { + assert_matches_golden("FALSE-MEMTRACK", "false_memtrack_child"); +} + +#[test] +fn false_memcleanup_child() { + if is_child() { + start(AnalysisMode::Memcleanup); + memory(); + map2check_success(); + } +} + +#[test] +fn false_memcleanup_matches_c_runtime() { + assert_matches_golden("FALSE-MEMCLEANUP", "false_memcleanup_child"); +} + +#[test] +fn none_child() { + if is_child() { + start(AnalysisMode::Memtrack); + memory(); + free_p(); + map2check_success(); + } +} + +#[test] +fn none_matches_c_runtime() { + assert_matches_golden("NONE", "none_child"); +} + +#[test] +fn unknown_child() { + if is_child() { + start(AnalysisMode::Memtrack); + memory(); + // SAFETY: a NULL block is only compared, never dereferenced. + unsafe { map2check_map_malloc(std::ptr::null(), 4) }; + map2check_success(); + } +} + +#[test] +fn unknown_matches_c_runtime() { + assert_matches_golden("UNKNOWN", "unknown_child"); +} + +#[test] +fn overflow_child() { + if is_child() { + start(AnalysisMode::Overflow); + // SAFETY: C-string literal is valid. + unsafe { map2check_binop_add_int(i32::MAX, 1, 20, 0, c"main".as_ptr()) }; + map2check_success(); + } +} + +#[test] +fn overflow_matches_c_runtime() { + assert_matches_golden("OVERFLOW", "overflow_child"); +} + +#[test] +fn assert_child() { + if is_child() { + start(AnalysisMode::Assert); + // SAFETY: C-string literal is valid. + unsafe { map2check_is_valid_assert(21, c"main".as_ptr(), 0) }; + map2check_success(); + } +} + +#[test] +fn assert_matches_c_runtime() { + assert_matches_golden("ASSERT", "assert_child"); +} + +#[test] +fn target_reached_child() { + if is_child() { + start(AnalysisMode::Target); + // SAFETY: C-string literal is valid. + unsafe { map2check_reach_target(c"reach_error".as_ptr(), 0, 22) }; + map2check_success(); + } +} + +#[test] +fn target_reached_matches_c_runtime() { + assert_matches_golden("TARGET-REACHED", "target_reached_child"); +} + +// ── Without witness: property files only (WitnessGenerationNone.c) ────────── + +#[test] +fn overflow_without_witness_child() { + if is_child() { + println!("{STARTED}"); + map2check_set_mode(AnalysisMode::Overflow as i32); + map2check_init(); + tracking(); + // SAFETY: C-string literal is valid. + unsafe { map2check_binop_add_int(i32::MAX, 1, 20, 0, c"main".as_ptr()) }; + println!("map2check-golden-child-survived"); + } +} + +#[test] +fn without_witness_a_violation_writes_only_property_files_and_aborts() { + let dir = tempfile::tempdir().unwrap(); + let output = run_child("overflow_without_witness_child", dir.path()); + assert_eq!(output.status.signal(), Some(6)); + assert!(!String::from_utf8_lossy(&output.stdout).contains("survived")); + assert_eq!( + files_in(dir.path()), + BTreeSet::from([ + "map2check_property".to_owned(), + "map2check_property_overflow".to_owned() + ]) + ); +} + +#[test] +fn success_without_witness_child() { + if is_child() { + println!("{STARTED}"); + map2check_set_mode(AnalysisMode::Memtrack as i32); + map2check_init(); + tracking(); + memory(); + free_p(); + map2check_success(); + println!("map2check-golden-child-returned"); + } +} + +#[test] +fn without_witness_success_writes_none_and_returns() { + let dir = tempfile::tempdir().unwrap(); + let output = run_child("success_without_witness_child", dir.path()); + assert_eq!(output.status.code(), Some(0)); + assert!(String::from_utf8_lossy(&output.stdout).contains("map2check-golden-child-returned")); + assert_eq!( + files_in(dir.path()), + BTreeSet::from(["map2check_property".to_owned()]) + ); + assert_eq!( + std::fs::read(dir.path().join("map2check_property")).unwrap(), + b"NONE\n" + ); +} + +// ── Stale property files and the unset mode ───────────────────────────────── + +#[test] +fn init_with_mode_removes_stale_property_files_child() { + if is_child() { + println!("{STARTED}"); + map2check_set_mode(AnalysisMode::Assert as i32); + map2check_init(); + } +} + +#[test] +fn init_with_mode_removes_stale_property_files() { + let dir = tempfile::tempdir().unwrap(); + for name in crate::output::legacy_files::PROPERTY_FILES { + std::fs::write(dir.path().join(name), "stale\n").unwrap(); + } + std::fs::write(dir.path().join("other"), "kept\n").unwrap(); + let output = run_child( + "init_with_mode_removes_stale_property_files_child", + dir.path(), + ); + assert_eq!(output.status.code(), Some(0)); + assert_eq!(files_in(dir.path()), BTreeSet::from(["other".to_owned()])); +} + +#[test] +fn unset_mode_records_without_files_or_abort_child() { + if is_child() { + println!("{STARTED}"); + map2check_enable_witness(); + map2check_init(); + // SAFETY: C-string literal is valid. + unsafe { map2check_binop_add_int(i32::MAX, 1, 20, 0, c"main".as_ptr()) }; + assert_eq!( + get_current_property(), + crate::caller::ViolatedProperty::Overflow + ); + map2check_success(); + println!("map2check-golden-child-returned"); + } +} + +#[test] +fn unset_mode_records_without_files_or_abort() { + let dir = tempfile::tempdir().unwrap(); + std::fs::write(dir.path().join("map2check_property_free"), "stale\n").unwrap(); + let output = run_child( + "unset_mode_records_without_files_or_abort_child", + dir.path(), + ); + assert_eq!(output.status.code(), Some(0)); + assert!(String::from_utf8_lossy(&output.stdout).contains("map2check-golden-child-returned")); + assert_eq!( + files_in(dir.path()), + BTreeSet::from(["map2check_property_free".to_owned()]) + ); +} diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index f53e9ed..02afdf4 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -1237,6 +1237,50 @@ fn options_survive_init_and_reset() { assert_eq!(observed_options(), expected); } +#[test] +fn enable_witness_is_configuration_kept_by_init_and_reset() { + let _guard = fresh(); + assert!(!state::config().witness); + map2check_enable_witness(); + assert!(state::config().witness); + map2check_init(); + map2check_reset(); + assert!(state::config().witness); + // Configuration only: it records nothing. + assert_ok(); + assert_eq!(snapshot(|s| s.current_step()), 0); +} + +#[test] +fn malloc_null_reports_unknown_in_memory_modes_only() { + // develop: map2check_malloc with a NULL block writes + // map2check_property__unknown and ends the run (AnalysisModeMemtrack.c:305-311). + let _guard = fresh(); + for mode in [AnalysisMode::Memtrack, AnalysisMode::Memcleanup] { + start(mode); + // SAFETY: a NULL block is only compared. + unsafe { map2check_map_malloc(std::ptr::null(), 4) }; + assert_violated(ViolatedProperty::Unknown, 0, ""); + assert!(snapshot(|s| s.memtrack.is_empty())); + start(mode); + // SAFETY: as above. + unsafe { map2check_map_calloc(std::ptr::null(), 2, 4) }; + assert_violated(ViolatedProperty::Unknown, 0, ""); + } + for mode in [AnalysisMode::Overflow, AnalysisMode::None] { + start(mode); + // SAFETY: as above. + unsafe { map2check_map_malloc(std::ptr::null(), 4) }; + assert_ok(); + assert_eq!(snapshot(|s| s.memtrack.len()), 1); + } + state::reset_config(); + state::reset(); + // SAFETY: as above. + unsafe { map2check_map_malloc(std::ptr::null(), 4) }; + assert_ok(); +} + #[test] fn options_default_off() { let _guard = fresh(); diff --git a/rust/src/memtrack/mod.rs b/rust/src/memtrack/mod.rs index 7f4d313..40c1024 100644 --- a/rust/src/memtrack/mod.rs +++ b/rust/src/memtrack/mod.rs @@ -26,6 +26,33 @@ pub struct MemTrackEntry { /// Size of the primitive element. pub size_primitive: usize, pub is_null_valid: bool, + /// The export that recorded the entry. Only the legacy log files read it + /// (`output::legacy_files`); the JSON report does not carry it. + #[serde(skip)] + pub origin: MemOrigin, +} + +/// Which export recorded a [`MemTrackEntry`]. The legacy log files of the +/// Map2Check `develop` runtime keep one log per kind of call (heap log, +/// allocation log, list log), so the replay that rebuilds them needs it. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub enum MemOrigin { + /// Built without an export (unit tests); the replay infers the kind + /// from the flags and the size. + #[default] + Unspecified, + /// `map2check_map_alloca` and `map2check_map_non_static_alloca`. + Alloca, + /// `map2check_map_funct_address`. + Function, + /// `map2check_map_malloc` and `map2check_map_calloc`. + Malloc, + /// `map2check_map_store_pointer`. + StorePointer, + /// `map2check_map_free`. + Free, + /// The release recorded by `map2check_check_free_resolved_address`. + FreeResolved, } impl MemTrackEntry { @@ -57,9 +84,16 @@ impl MemTrackEntry { size_destiny, size_primitive, is_null_valid, + origin: MemOrigin::Unspecified, } } + /// Sets the export that recorded the entry. + pub fn with_origin(mut self, origin: MemOrigin) -> Self { + self.origin = origin; + self + } + pub fn set_malloc(&mut self) { self.is_dynamic = true; self.is_free = false; diff --git a/rust/src/output/json_report.snap b/rust/src/output/json_report.snap new file mode 100644 index 0000000..fea6847 --- /dev/null +++ b/rust/src/output/json_report.snap @@ -0,0 +1,44 @@ +{ + "map2check_log": { + "result": "FALSE", + "step": 3, + "property": "MEMSAFETY_FREE", + "line_number": 12, + "function_name": "main", + "Container_TrackBBLog": [ + { + "step": 1, + "line": 5, + "function_name": "main" + } + ], + "Container_NonDetLog": [ + { + "step": 1, + "line": 10, + "scope": 0, + "function_name": "main", + "value": { + "type": "INT", + "value": 42 + } + } + ], + "Container_AllocationLog": [ + { + "step": 2, + "line": 12, + "scope": 1, + "var_mem_address": 256, + "mem_address_points_to": 512, + "is_dynamic": true, + "is_free": false, + "ptr_name": "p", + "function_name": "main", + "size_destiny": 16, + "size_primitive": 4, + "is_null_valid": false + } + ] + } +} diff --git a/rust/src/output/legacy_files.rs b/rust/src/output/legacy_files.rs new file mode 100644 index 0000000..f8a81d2 --- /dev/null +++ b/rust/src/output/legacy_files.rs @@ -0,0 +1,752 @@ +// SPDX-License-Identifier: GPL-3.0-only +// +// Legacy result channel (#19): the files that the C runtime of Map2Check +// `develop` writes into the current directory and that the Map2Check +// frontend reads (`modules/frontend/utils/tools.cpp`, `caller.cpp`, +// `counter_example/`). Every writer reproduces the `fprintf` calls of the C +// runtime byte for byte; `tests/legacy_golden/` holds the files that runtime +// wrote, and `rust/src/ffi_golden_tests.rs` compares against them. +// +// | file | develop writer | +// |---------------------------------|----------------------------------------| +// | `map2check_property*` | `PropertyGenerator.c` | +// | `map2check_checked_error` | `WitnessGeneration.c:28-32` | +// | `track_bb_log.st` | `TrackBBLog.c:35-44` | +// | `klee_log.csv` | `NonDetLog.c:17-60` | +// | `list_log.csv` | `ListLog.c:138-157` | +// | `allocation_log.csv` | `AllocationLog.c:161-174` | +// | `heap_log.csv` | `HeapLog.c:54-66` | +// +// The C runtime keeps one log per kind of record and its own step counter, +// which only list-log rows and nondet calls advance. This library keeps one +// memory log and one step counter for every record, so `LegacyLogs::replay` +// rebuilds develop's logs and steps from the records, in order, with the +// rules of `AnalysisModeMemtrack.c` / `AnalysisModeMemcleanup.c`. +#![deny(unsafe_code)] + +use std::fmt::Write as _; +use std::io; +use std::path::Path; + +use crate::{ + bbtrack::BasicBlockEntry, + caller::{AnalysisResult, ViolatedProperty}, + memtrack::{MemOrigin, MemTrackEntry}, + nondet::{NonDetEntry, NonDetValue}, +}; + +/// `PropertyGenerator.c:15-22`. `map2check_property__unknown` keeps the +/// double underscore of the C runtime on purpose (SD-16). +pub const PROPERTY_FILE: &str = "map2check_property"; +pub const PROPERTY_FILE_UNKNOWN: &str = "map2check_property__unknown"; +pub const PROPERTY_FILES: [&str; 8] = [ + PROPERTY_FILE, + PROPERTY_FILE_UNKNOWN, + "map2check_property_memtrack", + "map2check_property_memcleanup", + "map2check_property_deref", + "map2check_property_free", + "map2check_property_overflow", + "map2check_property_assert", +]; +pub const CHECKED_ERROR_FILE: &str = "map2check_checked_error"; +pub const TRACKBB_LOG_FILE: &str = "track_bb_log.st"; +pub const NONDET_LOG_FILE: &str = "klee_log.csv"; +pub const LIST_LOG_FILE: &str = "list_log.csv"; +pub const ALLOCATION_LOG_FILE: &str = "allocation_log.csv"; +pub const HEAP_LOG_FILE: &str = "heap_log.csv"; + +/// `%p` of glibc: `(nil)` for NULL, otherwise `0x` and lowercase hex. +pub fn c_pointer(address: usize) -> String { + if address == 0 { + "(nil)".to_owned() + } else { + format!("{address:#x}") + } +} + +/// `%lf` of glibc: six decimals, `inf`/`nan` with their sign. +pub fn c_double(value: f64) -> String { + let sign = if value.is_sign_negative() { "-" } else { "" }; + if value.is_nan() { + format!("{sign}nan") + } else if value.is_infinite() { + format!("{sign}inf") + } else { + format!("{value:.6}") + } +} + +/// Text of the verdict, as `write_property` prints it +/// (`PropertyGenerator.c:50-106`), and the extra per-property file the C +/// runtime writes next to `map2check_property`. `line` is printed with +/// `%d`, so it is shown as a C `int`. +fn property_text(result: &AnalysisResult) -> (Option, Option<&'static str>) { + let located = |tag: &str| { + format!( + "{tag}\nLine: {}\nFunction: {}\n", + result.line as i32, result.function_name + ) + }; + if result.ok { + return (Some("NONE\n".to_owned()), None); + } + match result.property { + ViolatedProperty::None => (Some("NONE\n".to_owned()), None), + ViolatedProperty::MemsafetyFree => { + (Some(located("FALSE-FREE")), Some("map2check_property_free")) + } + ViolatedProperty::MemsafetyDeref => ( + Some(located("FALSE-DEREF")), + Some("map2check_property_deref"), + ), + ViolatedProperty::Overflow => ( + Some(located("OVERFLOW")), + Some("map2check_property_overflow"), + ), + ViolatedProperty::Assert => (Some(located("ASSERT")), Some("map2check_property_assert")), + ViolatedProperty::MemsafetyMemtrack => ( + Some("FALSE-MEMTRACK\n".to_owned()), + Some("map2check_property_memtrack"), + ), + ViolatedProperty::MemsafetyMemcleanup => ( + Some("FALSE-MEMCLEANUP\n".to_owned()), + Some("map2check_property_memcleanup"), + ), + ViolatedProperty::Reachability => (Some(located("TARGET-REACHED")), None), + // The only UNKNOWN the C runtime reports is `write_property_unknown` + // (NULL from malloc, AnalysisModeMemtrack.c:305-311): it writes + // `map2check_property__unknown` and no `map2check_property`. + // Concurrency has no develop counterpart and is reported the same way. + ViolatedProperty::Unknown | ViolatedProperty::Concurrency => (None, None), + } +} + +/// Writes the property files of `result` into `dir`, as `write_property` +/// and the `write_property_*` helpers of `PropertyGenerator.c` do. +pub fn write_property_files(dir: &Path, result: &AnalysisResult) -> io::Result<()> { + let (text, extra) = property_text(result); + match text { + Some(text) => { + std::fs::write(dir.join(PROPERTY_FILE), &text)?; + if let Some(extra) = extra { + std::fs::write(dir.join(extra), &text)?; + } + } + None => std::fs::write(dir.join(PROPERTY_FILE_UNKNOWN), "UNKNOWN\n")?, + } + Ok(()) +} + +/// Removes every property file left in `dir` by an earlier run. A missing +/// file is not an error; other errors are reported on stderr. +pub fn remove_stale_property_files(dir: &Path) { + for name in PROPERTY_FILES { + match std::fs::remove_file(dir.join(name)) { + Ok(()) => {} + Err(e) if e.kind() == io::ErrorKind::NotFound => {} + Err(e) => eprintln!("[map2check] cannot remove stale {name}: {e}"), + } + } +} + +/// `generate_confirmation_file` (`WitnessGeneration.c:28-32`). +pub fn write_checked_error(dir: &Path) -> io::Result<()> { + std::fs::write(dir.join(CHECKED_ERROR_FILE), "END OF TIME\n") +} + +/// `trackbb_log_to_file` (`TrackBBLog.c:35-44`): `%u;%s\n` per row. +/// `map2check_track_bb` keeps only the first block of each line +/// (`Map2CheckFunctions.c:49-55`), so later rows of a line are skipped. +pub fn write_trackbb_log(dir: &Path, rows: &[BasicBlockEntry]) -> io::Result<()> { + let mut seen = std::collections::HashSet::new(); + let mut out = String::new(); + for row in rows.iter().filter(|r| seen.insert(r.line)) { + let _ = writeln!(out, "{};{}", row.line, row.function_name); + } + std::fs::write(dir.join(TRACKBB_LOG_FILE), out) +} + +/// The value column and the `enum NONDET_TYPE` number +/// (`Map2CheckTypes.h:41-58`) of a nondet row, as `nondet_log_to_file` +/// prints them (`NonDetLog.c:45-54`). The `map2check_nondet_*` wrappers take +/// the value as a C `int` (`NonDetLog.c:105-113`), so every integer kind is +/// printed as its low 32 bits with `%d`; a double is printed with `%lf`. +/// `float` has no develop counterpart and is logged as a double. +pub fn nondet_value_and_type(value: &NonDetValue) -> (String, i32) { + match *value { + NonDetValue::Int(v) => (v.to_string(), 0), + NonDetValue::Char(v) => ((v as i32).to_string(), 1), + NonDetValue::Long(v) => ((v as i32).to_string(), 4), + NonDetValue::UInt(v) => ((v as i32).to_string(), 13), + NonDetValue::Float(v) => (c_double(f64::from(v)), 15), + NonDetValue::Double(v) => (c_double(v), 15), + } +} + +/// `enum MemoryAddressStatus` (`Map2CheckTypes.h:34`). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Status { + Static, + Free, + Dynamic, +} + +impl Status { + /// `(is_dynamic, is_free)` of a list row with this status + /// (`AnalysisModeMemtrack.c:217-230`, `:256-272`). + fn flags(self) -> (bool, bool) { + match self { + Status::Static => (false, false), + Status::Free => (false, true), + Status::Dynamic => (true, false), + } + } +} + +#[derive(Debug, Clone)] +struct ListRow { + var: usize, + points_to: usize, + scope: u32, + is_dynamic: bool, + is_free: bool, + line: u32, + name: String, + function: String, + step: u32, +} + +impl ListRow { + /// `get_type_from_list_log_row` (`ListLog.c:159-168`). + fn status(&self) -> Status { + if self.is_free && !self.is_dynamic { + Status::Free + } else if self.is_dynamic { + Status::Dynamic + } else { + Status::Static + } + } +} + +#[derive(Debug, Clone)] +struct AllocationRow { + addr: usize, + size: u32, + is_free: bool, +} + +#[derive(Debug, Clone)] +struct HeapRow { + addr: usize, + name: String, + size: i32, +} + +#[derive(Debug, Clone)] +struct NondetRow { + line: u32, + scope: u32, + function: String, + step: u32, + value: NonDetValue, +} + +/// develop's heap, allocation, list and nondet logs, rebuilt from the +/// records of this library. +#[derive(Debug, Default)] +pub struct LegacyLogs { + list: Vec, + allocation: Vec, + heap: Vec, + nondet: Vec, + /// `Map2CheckCurrentStep` (`Map2CheckFunctions.c:19`). + step: u32, +} + +/// One record, memory or nondet, in the order the program made it. +enum Event<'a> { + Memory(&'a MemTrackEntry), + Nondet(&'a NonDetEntry), +} + +impl LegacyLogs { + /// Replays the memory and nondet records in the order the program made + /// them (both carry this library's step). `end_of_program` adds what + /// `valid_allocation_log` appends when the end-of-program verdict runs + /// (`AllocationLog.c:69-101`). + pub fn replay( + memtrack: &[MemTrackEntry], + nondets: &[NonDetEntry], + end_of_program: bool, + ) -> Self { + let mut events: Vec<(u64, Event<'_>)> = memtrack + .iter() + .map(|e| (e.step, Event::Memory(e))) + .chain(nondets.iter().map(|e| (e.step, Event::Nondet(e)))) + .collect(); + events.sort_by_key(|(step, _)| *step); + + let mut logs = Self::default(); + for (_, event) in events { + match event { + Event::Memory(e) => logs.memory(e), + Event::Nondet(e) => logs.nondet(e), + } + } + if end_of_program { + logs.valid_allocation_log(); + } + logs + } + + fn next_step(&mut self) -> u32 { + let step = self.step; + self.step = self.step.wrapping_add(1); + step + } + + fn memory(&mut self, e: &MemTrackEntry) { + let origin = match e.origin { + MemOrigin::Unspecified if e.is_free => MemOrigin::FreeResolved, + MemOrigin::Unspecified if e.is_dynamic => MemOrigin::Malloc, + MemOrigin::Unspecified if e.size_destiny == 0 => MemOrigin::StorePointer, + MemOrigin::Unspecified => MemOrigin::Alloca, + origin => origin, + }; + match origin { + // map2check_alloca (AnalysisModeMemtrack.c:374-379). + MemOrigin::Alloca => self.heap.push(HeapRow { + addr: e.var_addr, + name: e.ptr_name.clone(), + size: e.size_destiny as i32, + }), + // map2check_function (:381-385): size 1. + MemOrigin::Function => self.heap.push(HeapRow { + addr: e.var_addr, + name: e.ptr_name.clone(), + size: 1, + }), + MemOrigin::Malloc => self.malloc(e.points_to, e.size_destiny as u32), + MemOrigin::StorePointer => self.store_pointer(e), + MemOrigin::Free => self.free(e), + MemOrigin::FreeResolved => self.free_resolved(e.points_to, e.line), + MemOrigin::Unspecified => {} + } + } + + /// `map2check_nondet_*` (`NonDetLog.c:96-113`): the row takes the + /// current step, then the step advances. + fn nondet(&mut self, e: &NonDetEntry) { + let step = self.next_step(); + self.nondet.push(NondetRow { + line: e.line, + scope: e.scope, + function: e.function_name.clone(), + step, + value: e.value.clone(), + }); + } + + /// `map2check_malloc` (`AnalysisModeMemtrack.c:305-323`). A NULL block + /// ends the develop run before any row (UNKNOWN). + fn malloc(&mut self, addr: usize, size: u32) { + if addr == 0 { + return; + } + match self.allocation.iter_mut().rev().find(|r| r.addr == addr) { + Some(row) => { + row.size = size; + row.is_free = false; + } + None => self.allocation.push(AllocationRow { + addr, + size, + is_free: false, + }), + } + } + + /// `check_address_allocation_log` (`AllocationLog.c:34-51`). + fn status_of(&self, addr: usize) -> Status { + match self.allocation.iter().rev().find(|r| r.addr == addr) { + Some(r) if r.is_free => Status::Free, + Some(_) => Status::Dynamic, + None => Status::Static, + } + } + + /// `map2check_add_store_pointer` (`AnalysisModeMemtrack.c:246-295`). + fn store_pointer(&mut self, e: &MemTrackEntry) { + let status = self.status_of(e.points_to); + let (is_dynamic, is_free) = status.flags(); + let redundant = self + .list + .iter() + .rev() + .find(|r| r.var == e.var_addr) + .is_some_and(|r| { + r.points_to == e.points_to && r.is_free == is_free && r.is_dynamic == is_dynamic + }); + if redundant { + return; + } + let step = self.next_step(); + self.list.push(ListRow { + var: e.var_addr, + points_to: e.points_to, + scope: e.scope, + is_dynamic, + is_free, + line: e.line, + name: e.ptr_name.clone(), + function: e.function_name.clone(), + step, + }); + self.update_reference(e.points_to, status, e.line); + } + + /// `map2check_free` (`AnalysisModeMemtrack.c:339-366`): a FREE row for + /// the pointer variable, then the block is released. The C runtime reads + /// the variable's address from its argument; this library's free record + /// has only the block, so the variable is the one of the newest list row + /// that points to the block (SD-17). + fn free(&mut self, e: &MemTrackEntry) { + let addr = e.points_to; + if addr == 0 { + return; + } + let var = self + .list + .iter() + .rev() + .find(|r| r.points_to == addr) + .map_or(addr, |r| r.var); + let step = self.next_step(); + self.list.push(ListRow { + var, + points_to: addr, + scope: e.scope, + is_dynamic: false, + is_free: true, + line: e.line, + name: e.ptr_name.clone(), + function: e.function_name.clone(), + step, + }); + self.release(addr, e.line); + } + + /// `map2check_free_resolved_address` (`AnalysisModeMemtrack.c:164-183`). + fn free_resolved(&mut self, addr: usize, line: u32) { + if addr == 0 { + return; + } + self.release(addr, line); + } + + /// The allocation row becomes free with size 0 and every pointer to the + /// block gets a FREE row. + fn release(&mut self, addr: usize, line: u32) { + if let Some(row) = self.allocation.iter_mut().rev().find(|r| r.addr == addr) { + row.size = 0; + row.is_free = true; + } + self.update_reference(addr, Status::Free, line); + } + + /// `update_reference_list_log` (`AnalysisModeMemtrack.c:185-244`). + fn update_reference(&mut self, address: usize, status: Status, line: u32) { + let current_size = self.list.len(); + for i in 0..current_size { + let (i_var, i_points_to, i_status) = { + let r = &self.list[i]; + (r.var, r.points_to, r.status()) + }; + if i_points_to != address || i_status == status { + continue; + } + let Some(j_row) = self.list[i..current_size] + .iter() + .rev() + .find(|r| r.points_to == address && r.var == i_var) + .cloned() + else { + continue; + }; + if j_row.status() != status { + let (is_dynamic, is_free) = status.flags(); + let step = self.next_step(); + self.list.push(ListRow { + is_dynamic, + is_free, + line, + step, + ..j_row + }); + } + } + } + + /// `valid_allocation_log` (`AllocationLog.c:69-101`): every block with + /// no later freed row gets a freed row of size 0 appended. + fn valid_allocation_log(&mut self) { + let mut i = 0; + while i < self.allocation.len() { + let row = &self.allocation[i]; + if !row.is_free { + let addr = row.addr; + let released = self.allocation[i + 1..] + .iter() + .any(|r| r.addr == addr && r.is_free); + if !released { + self.allocation.push(AllocationRow { + addr, + size: 0, + is_free: true, + }); + } + } + i += 1; + } + } + + /// `list_log_to_file` (`ListLog.c:138-157`). The id column is always 0 + /// (`new_list_row`, `ListLog.c:125`). + pub fn write_list_log(&self, dir: &Path) -> io::Result<()> { + let mut out = String::new(); + for r in &self.list { + let _ = writeln!( + out, + "0;{};{};{};{};{};{};{};{};{}", + c_pointer(r.var), + c_pointer(r.points_to), + r.scope, + i32::from(r.is_free), + i32::from(r.is_dynamic), + r.name, + r.line, + r.function, + r.step + ); + } + std::fs::write(dir.join(LIST_LOG_FILE), out) + } + + /// `allocation_log_to_file` (`AllocationLog.c:161-174`). + pub fn write_allocation_log(&self, dir: &Path) -> io::Result<()> { + let mut out = String::new(); + for r in &self.allocation { + let _ = writeln!( + out, + "{};{};{}", + c_pointer(r.addr), + r.size, + i32::from(r.is_free) + ); + } + std::fs::write(dir.join(ALLOCATION_LOG_FILE), out) + } + + /// `heap_log_to_file` (`HeapLog.c:54-66`). + pub fn write_heap_log(&self, dir: &Path) -> io::Result<()> { + let mut out = String::new(); + for r in &self.heap { + let _ = writeln!(out, "{};{};{}", c_pointer(r.addr), r.name, r.size); + } + std::fs::write(dir.join(HEAP_LOG_FILE), out) + } + + /// `nondet_log_to_file` (`NonDetLog.c:17-60`): + /// `index;line;scope;function;step;value;type`. + pub fn write_nondet_log(&self, dir: &Path) -> io::Result<()> { + let mut out = String::new(); + for (i, r) in self.nondet.iter().enumerate() { + let (value, kind) = nondet_value_and_type(&r.value); + let _ = writeln!( + out, + "{i};{};{};{};{};{value};{kind}", + r.line, r.scope, r.function, r.step + ); + } + std::fs::write(dir.join(NONDET_LOG_FILE), out) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::caller::{AnalysisResult, ViolatedProperty}; + + fn golden(tag: &str, file: &str) -> Vec { + std::fs::read(format!( + "{}/tests/legacy_golden/{tag}/{file}", + env!("CARGO_MANIFEST_DIR") + )) + .unwrap() + } + + fn violated(property: ViolatedProperty, line: u32, function_name: &str) -> AnalysisResult { + AnalysisResult { + ok: false, + property, + line, + function_name: function_name.into(), + ..Default::default() + } + } + + #[test] + fn property_files_match_c_runtime_for_every_tag() { + let cases = [ + ( + "FALSE-FREE", + violated(ViolatedProperty::MemsafetyFree, 13, "main"), + "map2check_property_free", + ), + ( + "FALSE-DEREF", + violated(ViolatedProperty::MemsafetyDeref, 14, "main"), + "map2check_property_deref", + ), + ( + "FALSE-MEMTRACK", + violated(ViolatedProperty::MemsafetyMemtrack, 0, ""), + "map2check_property_memtrack", + ), + ( + "FALSE-MEMCLEANUP", + violated(ViolatedProperty::MemsafetyMemcleanup, 0, ""), + "map2check_property_memcleanup", + ), + ( + "OVERFLOW", + violated(ViolatedProperty::Overflow, 20, "main"), + "map2check_property_overflow", + ), + ( + "ASSERT", + violated(ViolatedProperty::Assert, 21, "main"), + "map2check_property_assert", + ), + ]; + for (tag, result, extra) in cases { + let dir = tempfile::tempdir().unwrap(); + write_property_files(dir.path(), &result).unwrap(); + for file in ["map2check_property", extra] { + assert_eq!( + std::fs::read(dir.path().join(file)).unwrap(), + golden(tag, file), + "{tag}/{file}" + ); + } + assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 2, "{tag}"); + } + } + + #[test] + fn target_reached_and_none_write_only_map2check_property() { + for (tag, result) in [ + ( + "TARGET-REACHED", + violated(ViolatedProperty::Reachability, 22, "reach_error"), + ), + ("NONE", AnalysisResult::default()), + ] { + let dir = tempfile::tempdir().unwrap(); + write_property_files(dir.path(), &result).unwrap(); + assert_eq!( + std::fs::read(dir.path().join("map2check_property")).unwrap(), + golden(tag, "map2check_property"), + "{tag}" + ); + assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 1, "{tag}"); + } + } + + #[test] + fn unknown_uses_legacy_double_underscore_name() { + // PropertyGenerator.c:16; SD-16. + let dir = tempfile::tempdir().unwrap(); + write_property_files(dir.path(), &violated(ViolatedProperty::Unknown, 0, "")).unwrap(); + assert_eq!( + std::fs::read(dir.path().join("map2check_property__unknown")).unwrap(), + golden("UNKNOWN", "map2check_property__unknown") + ); + assert!(!dir.path().join("map2check_property_unknown").exists()); + assert!(!dir.path().join("map2check_property").exists()); + } + + #[test] + fn stale_property_files_are_removed() { + let dir = tempfile::tempdir().unwrap(); + for name in PROPERTY_FILES { + std::fs::write(dir.path().join(name), "stale\n").unwrap(); + } + std::fs::write(dir.path().join("track_bb_log.st"), "3;main\n").unwrap(); + remove_stale_property_files(dir.path()); + let left: Vec<_> = std::fs::read_dir(dir.path()) + .unwrap() + .map(|e| e.unwrap().file_name()) + .collect(); + assert_eq!(left, ["track_bb_log.st"]); + // Nothing to remove is not an error. + remove_stale_property_files(dir.path()); + } + + #[test] + fn checked_error_matches_c_runtime() { + let dir = tempfile::tempdir().unwrap(); + write_checked_error(dir.path()).unwrap(); + assert_eq!( + std::fs::read(dir.path().join("map2check_checked_error")).unwrap(), + golden("ASSERT", "map2check_checked_error") + ); + } + + #[test] + fn trackbb_log_keeps_the_first_row_of_each_line() { + use crate::bbtrack::BasicBlockEntry; + let dir = tempfile::tempdir().unwrap(); + let rows = [ + BasicBlockEntry::new(1, 3, "main"), + BasicBlockEntry::new(2, 4, "main"), + BasicBlockEntry::new(3, 3, "other"), + ]; + write_trackbb_log(dir.path(), &rows).unwrap(); + assert_eq!( + std::fs::read(dir.path().join("track_bb_log.st")).unwrap(), + golden("ASSERT", "track_bb_log.st") + ); + } + + #[test] + fn c_formats_match_glibc() { + assert_eq!(c_pointer(0), "(nil)"); + assert_eq!(c_pointer(0x1000_0100), "0x10000100"); + assert_eq!(c_double(2.5), "2.500000"); + assert_eq!(c_double(-0.0), "-0.000000"); + assert_eq!(c_double(f64::INFINITY), "inf"); + assert_eq!(c_double(f64::NEG_INFINITY), "-inf"); + assert_eq!(c_double(f64::NAN), "nan"); + assert_eq!(c_double(-f64::NAN), "-nan"); + } + + #[test] + fn nondet_rows_use_develop_types_and_value_formats() { + use crate::nondet::NonDetValue as V; + assert_eq!(nondet_value_and_type(&V::Int(-1)), ("-1".into(), 0)); + assert_eq!(nondet_value_and_type(&V::UInt(u32::MAX)), ("-1".into(), 13)); + assert_eq!(nondet_value_and_type(&V::Long(1 << 32)), ("0".into(), 4)); + assert_eq!(nondet_value_and_type(&V::Char(-3)), ("-3".into(), 1)); + assert_eq!( + nondet_value_and_type(&V::Float(0.5)), + ("0.500000".into(), 15) + ); + assert_eq!( + nondet_value_and_type(&V::Double(2.5)), + ("2.500000".into(), 15) + ); + } +} diff --git a/rust/src/output/mod.rs b/rust/src/output/mod.rs index 88e9c36..efa0276 100644 --- a/rust/src/output/mod.rs +++ b/rust/src/output/mod.rs @@ -4,6 +4,9 @@ // Replaces lib/json-maker.c and lib/to_string/ftoa.c entirely. // serde_json handles JSON construction and float formatting safely. +pub mod legacy_files; +pub(crate) mod termination; + use serde::Serialize; use crate::{caller::ViolatedProperty, error::Map2CheckError, state::AnalysisState}; @@ -155,6 +158,28 @@ mod tests { assert!(parsed["map2check_log"].is_object()); } + /// Snapshot of the whole report (#14): the JSON is an additional channel + /// with its own schema (SD-01); Map2Check reads the files of + /// `legacy_files`. A change to this text is a schema change. + #[test] + fn json_report_snapshot() { + let mut state = state_ok(); + state.result.ok = false; + state.result.step = 3; + state.result.property = ViolatedProperty::MemsafetyFree; + state.result.line = 12; + state.result.function_name = "main".to_owned(); + state.memtrack.push(crate::memtrack::MemTrackEntry::new( + 2, 12, 1, 0x100, 0x200, true, false, "p", "main", 16, 4, false, + )); + let snapshot = std::fs::read_to_string(concat!( + env!("CARGO_MANIFEST_DIR"), + "/rust/src/output/json_report.snap" + )) + .unwrap(); + assert_eq!(to_json(&state).unwrap() + "\n", snapshot); + } + #[test] fn print_json_succeeds_for_valid_state() { assert!(print_json(&state_ok()).is_ok()); diff --git a/rust/src/output/termination.rs b/rust/src/output/termination.rs new file mode 100644 index 0000000..9534614 --- /dev/null +++ b/rust/src/output/termination.rs @@ -0,0 +1,176 @@ +// SPDX-License-Identifier: GPL-3.0-only +// +// End of a Map2Check run (#19): the files the frontend reads and the abort +// that tells it a violation was found. Mirrors `map2check_success`, +// `map2check_error`, `map2check_exit` and `map2check_destroy` of the C +// runtime of Map2Check `develop` (`Map2CheckFunctions.c:67-102`). +// +// Only a Map2Check-instrumented program selects an analysis mode +// (`map2check_set_mode`), so the mode decides: with a mode set, the first +// violation writes the files and aborts, and `map2check_success` writes the +// files and returns; with the mode unset (direct API users, the C harness, +// unit tests) nothing is written and nothing ends the process. +#![deny(unsafe_code)] + +use std::path::{Path, PathBuf}; + +use super::legacy_files::{self, LegacyLogs}; +use crate::{ + analysismode::mode::{self, AnalysisMode}, + caller::AnalysisResult, + state::{self, AnalysisState, Config}, +}; + +/// Test builds write into this directory, and only when it is set; the +/// golden tests set it for a child process (`rust/src/ffi_golden_tests.rs`). +#[cfg(test)] +pub(crate) const OUTPUT_DIR_ENV: &str = "MAP2CHECK_TEST_OUTPUT_DIR"; + +/// Where the result files go: the current directory, as in develop. In test +/// builds, the directory named by [`OUTPUT_DIR_ENV`], or nowhere, so that a +/// unit test with a mode set records the violation and keeps running. +pub(crate) fn output_dir() -> Option { + #[cfg(not(test))] + { + Some(PathBuf::from(".")) + } + #[cfg(test)] + { + std::env::var_os(OUTPUT_DIR_ENV).map(PathBuf::from) + } +} + +/// The directory to write into, when this run uses the file channel: a mode +/// is set and there is an output directory. +fn channel(config: Config) -> Option { + config.mode.and(output_dir()) +} + +fn report(what: &str, result: std::io::Result<()>) { + if let Err(e) = result { + eprintln!("[map2check] cannot write {what}: {e}"); + } +} + +/// `map2check_init()` with a mode set: removes the property files of an +/// earlier run before this one writes its own. +pub(crate) fn on_init() { + if let Some(dir) = channel(state::config()) { + legacy_files::remove_stale_property_files(&dir); + } +} + +/// `generate_aux_files` (`WitnessGeneration.c:14-25`) when witness +/// generation is on: the basic-block log, the logs of the memory modes +/// (`analysis_generate_aux_witness_files`, `AnalysisModeMemtrack.c:148-152`) +/// and, on a violation, the nondet log. +fn write_aux_files(dir: &Path, config: Config, s: &AnalysisState, end_of_program: bool) { + report( + legacy_files::TRACKBB_LOG_FILE, + legacy_files::write_trackbb_log(dir, &s.bbtrack), + ); + let logs = LegacyLogs::replay(&s.memtrack, &s.nondets, end_of_program); + if matches!( + config.mode, + Some(AnalysisMode::Memtrack | AnalysisMode::Memcleanup) + ) { + report(legacy_files::HEAP_LOG_FILE, logs.write_heap_log(dir)); + report( + legacy_files::ALLOCATION_LOG_FILE, + logs.write_allocation_log(dir), + ); + report(legacy_files::LIST_LOG_FILE, logs.write_list_log(dir)); + } + if !s.result.ok { + report(legacy_files::NONDET_LOG_FILE, logs.write_nondet_log(dir)); + } +} + +/// `map2check_exit` after the verdict is known: aux files, then +/// `analysis_destroy`, then, on a violation, the confirmation file and the +/// abort. Returns only when there is no violation. +fn exit(dir: &Path, config: Config, end_of_program: bool) { + let failed = state::with_state(|s| { + if config.witness { + write_aux_files(dir, config, s, end_of_program); + } + mode::analysis_destroy(config.mode, s); + Ok(!s.result.ok) + }) + .unwrap_or_else(|e| { + eprintln!("[map2check] error: {e}"); + true + }); + if failed { + if config.witness { + report( + legacy_files::CHECKED_ERROR_FILE, + legacy_files::write_checked_error(dir), + ); + } + std::process::abort(); + } +} + +fn write_properties(dir: &Path, result: &AnalysisResult) { + report( + "the property files", + legacy_files::write_property_files(dir, result), + ); +} + +/// The first violation was just recorded (`map2check_error`, +/// `Map2CheckFunctions.c:72-75`): with a mode set, writes the property +/// files, ends the run and aborts. `end_of_program` is true when the +/// violation is the end-of-program verdict, whose check appends rows to +/// develop's allocation log. Without a mode, returns at once. +pub(crate) fn on_violation(end_of_program: bool) { + let config = state::config(); + let Some(dir) = channel(config) else { + return; + }; + match state::with_state(|s| Ok(s.result.clone())) { + Ok(result) => write_properties(&dir, &result), + Err(e) => eprintln!("[map2check] error: {e}"), + } + exit(&dir, config, end_of_program); + // `exit` returns only without a violation, which a recorded violation + // rules out; abort anyway rather than let the program continue. + std::process::abort(); +} + +/// `map2check_success` (`Map2CheckFunctions.c:67-70`, `:89-102`): with a +/// mode set, writes `NONE`, runs the end-of-program verdict, ends the run, +/// and returns when the program is correct. Returns whether the file +/// channel handled the call. +pub(crate) fn on_success(after_verdict: impl FnOnce(&AnalysisState)) -> bool { + let config = state::config(); + let Some(dir) = channel(config) else { + return false; + }; + write_properties(&dir, &AnalysisResult::default()); + let verdict = state::with_state(|s| { + let end_of_program = s.result.ok; + if end_of_program { + if let Some(property) = mode::analysis_is_program_correct(config, s) { + s.set_false(property, 0, ""); + } + } + after_verdict(s); + Ok((end_of_program, s.result.clone())) + }); + let end_of_program = match verdict { + Ok((end_of_program, result)) => { + if !result.ok { + write_properties(&dir, &result); + } + end_of_program + } + Err(e) => { + eprintln!("[map2check] error: {e}"); + false + } + }; + exit(&dir, config, end_of_program); + true +} diff --git a/rust/src/state.rs b/rust/src/state.rs index 0811a06..77afdee 100644 --- a/rust/src/state.rs +++ b/rust/src/state.rs @@ -30,6 +30,11 @@ pub struct Config { pub memcleanup: bool, /// Set by `map2check_set_null_is_valid()`: a NULL load is safe. pub null_is_valid: bool, + /// Set by `map2check_enable_witness()`: the end of a Map2Check run also + /// writes the witness files (`track_bb_log.st`, the memory logs, + /// `klee_log.csv` and `map2check_checked_error`), as a develop binary + /// linked with `WitnessGeneration.c` does. + pub witness: bool, } impl Config { @@ -38,6 +43,7 @@ impl Config { mode: None, memcleanup: false, null_is_valid: false, + witness: false, } } } diff --git a/tests/compat_docs.rs b/tests/compat_docs.rs index 350356b..f7545cb 100644 --- a/tests/compat_docs.rs +++ b/tests/compat_docs.rs @@ -322,6 +322,7 @@ const CLASSIFICATIONS: &[&str] = &[ "suspected regression", "pending decision", "fixed", + "C behaviour preserved on purpose", ]; /// Problemas de uma entrada: classificação conhecida (com issue quando diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index 4a95962..340ebd0 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -11,9 +11,14 @@ * does not expand `macro_rules!`. Those are declared by hand below with the * exact signatures from `ffi.rs`. */ +#include #include #include +#include #include +#include +#include +#include #include "../include/map2check.h" @@ -352,63 +357,192 @@ static int test_result_and_control_flags(void) { return 0; } -/* Task 2.3: the mode selected at run time gates the checks. Runs last - * because no API returns the mode to unset. */ -static int test_analysis_modes(void) { +/* Task 2.3 and 2.4: with a mode set, the first violation writes the result + * files and aborts, and map2check_success() writes NONE and returns, as the + * C runtime of Map2Check develop does. Each scenario therefore runs in a + * forked child inside its own temporary directory; the parent checks how + * the child ended and what it wrote. The mode stays unset in this process. */ +typedef void (*scenario_fn)(void); + +static const char *const result_files[] = { + "map2check_property", "map2check_property__unknown", + "map2check_property_memtrack", "map2check_property_memcleanup", + "map2check_property_deref", "map2check_property_free", + "map2check_property_overflow", "map2check_property_assert", + "map2check_checked_error", "track_bb_log.st", "klee_log.csv", + "list_log.csv", "allocation_log.csv", "heap_log.csv", NULL}; + +/* Reads `name` from `dir` into `buf` (NUL-terminated); "" when missing. */ +static void read_result(const char *dir, const char *name, char *buf, + size_t size) { + char path[512]; + snprintf(path, sizeof(path), "%s/%s", dir, name); + buf[0] = '\0'; + FILE *f = fopen(path, "r"); + if (f == NULL) { + return; + } + size_t n = fread(buf, 1, size - 1, f); + buf[n] = '\0'; + fclose(f); +} + +static void remove_results(const char *dir) { + char path[512]; + for (int i = 0; result_files[i] != NULL; i++) { + snprintf(path, sizeof(path), "%s/%s", dir, result_files[i]); + unlink(path); + } + rmdir(dir); +} + +/* Runs `scenario` in a child. `aborts`: whether it must end with SIGABRT + * (otherwise exit 0). `property`: expected map2check_property content. + * `extra`/`extra_content`: one more file to check, or NULL. */ +static int run_mode_scenario(scenario_fn scenario, int aborts, + const char *property, const char *extra, + const char *extra_content) { + char dir[] = "/tmp/map2check-harness-XXXXXX"; + if (mkdtemp(dir) == NULL) { + return 1; + } + fflush(NULL); + pid_t pid = fork(); + if (pid < 0) { + return 1; + } + if (pid == 0) { + if (chdir(dir) != 0) { + _exit(90); + } + scenario(); + _exit(0); + } + int status = 0; + int failed = waitpid(pid, &status, 0) != pid; + if (aborts) { + failed |= !(WIFSIGNALED(status) && WTERMSIG(status) == SIGABRT); + } else { + failed |= !(WIFEXITED(status) && WEXITSTATUS(status) == 0); + } + char buf[256]; + read_result(dir, "map2check_property", buf, sizeof(buf)); + failed |= strcmp(buf, property) != 0; + if (extra != NULL) { + read_result(dir, extra, buf, sizeof(buf)); + failed |= strcmp(buf, extra_content) != 0; + } + remove_results(dir); + return failed; +} + +static void scenario_none_mode(void) { map2check_set_mode(AnalysisMode_None); map2check_init(); map2check_check_deref((void *)0xdead, 0, 1, "main"); if (get_current_property() != None) { - return 1; + _exit(91); } + map2check_success(); +} +static void scenario_memtrack_deref(void) { map2check_set_mode(AnalysisMode_Memtrack); map2check_init(); map2check_check_deref((void *)0xdead, 0, 1, "main"); - if (get_current_property() != MemsafetyDeref) { - return 2; - } +} + +static void scenario_memtrack_deref_witness(void) { + map2check_enable_witness(); + scenario_memtrack_deref(); +} +static void scenario_assert_ignores_overflow(void) { map2check_set_mode(AnalysisMode_Assert); map2check_init(); map2check_binop_add_int(2147483647, 1, 2, 0, "main"); if (get_current_property() != None) { - return 3; + _exit(91); } + map2check_success(); +} +static void scenario_target(void) { map2check_set_mode(AnalysisMode_Target); map2check_init(); map2check_reach_target("reach_error", 0, 12); - if (get_current_property() != Reachability) { - return 4; - } +} - /* Out-of-range values are ignored: the mode stays Target (I-1). */ +/* Out-of-range values are ignored: the mode stays Target (I-1). */ +static void scenario_out_of_range_keeps_target(void) { + map2check_set_mode(AnalysisMode_Target); map2check_set_mode(-1); map2check_set_mode(6); map2check_set_mode(2147483647); map2check_init(); map2check_check_deref((void *)0xdead, 0, 1, "main"); if (get_current_property() != None) { - return 5; + _exit(91); } + map2check_success(); +} - /* Memcleanup mode: only a leak still referenced by a pointer (I-2). */ +/* Memcleanup mode: an unreferenced leak gives no verdict (I-2). */ +static void scenario_memcleanup_unreferenced(void) { map2check_set_mode(AnalysisMode_Memcleanup); map2check_init(); map2check_map_malloc((void *)0x3000, 8); map2check_check_mem_endprog(); if (get_current_property() != None) { - return 6; + _exit(91); } + map2check_success(); +} + +static void scenario_memcleanup_referenced(void) { + map2check_set_mode(AnalysisMode_Memcleanup); map2check_init(); map2check_map_malloc((void *)0x3000, 8); map2check_map_store_pointer((void *)0x10, (void *)0x3000, 0, "p", 1, "main"); map2check_check_mem_endprog(); - if (get_current_property() != MemsafetyMemcleanup) { +} + +static int test_analysis_modes(void) { + if (run_mode_scenario(scenario_none_mode, 0, "NONE\n", NULL, NULL)) { + return 1; + } + if (run_mode_scenario(scenario_memtrack_deref, 1, + "FALSE-DEREF\nLine: 1\nFunction: main\n", + "map2check_checked_error", "")) { + return 2; + } + if (run_mode_scenario(scenario_memtrack_deref_witness, 1, + "FALSE-DEREF\nLine: 1\nFunction: main\n", + "map2check_checked_error", "END OF TIME\n")) { + return 3; + } + if (run_mode_scenario(scenario_assert_ignores_overflow, 0, "NONE\n", NULL, + NULL)) { + return 4; + } + if (run_mode_scenario(scenario_target, 1, + "TARGET-REACHED\nLine: 12\nFunction: reach_error\n", + NULL, NULL)) { + return 5; + } + if (run_mode_scenario(scenario_out_of_range_keeps_target, 0, "NONE\n", + NULL, NULL)) { + return 6; + } + if (run_mode_scenario(scenario_memcleanup_unreferenced, 0, "NONE\n", NULL, + NULL)) { return 7; } - map2check_reset(); + if (run_mode_scenario(scenario_memcleanup_referenced, 1, + "FALSE-MEMCLEANUP\n", "map2check_property_memcleanup", + "FALSE-MEMCLEANUP\n")) { + return 8; + } return 0; } @@ -448,8 +582,13 @@ int main(void) { } } - if (test_analysis_modes() != 0) { - return 1; + { + int failed_mode = test_analysis_modes(); + if (failed_mode != 0) { + fprintf(stderr, "test_analysis_modes: scenario %d failed\n", + failed_mode); + return 1; + } } map2check_success(); From a62b4641cadc4da91855dbea8ab9dda1c5ed56e8 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 22:24:31 -0400 Subject: [PATCH 25/63] fix: print no JSON with an analysis mode set and end an unreadable run as UNKNOWN (#19) With a mode set no export writes to stdout, as in develop: map2check_success, print_all_containers_as_json, a failing map2check_is_valid_assert and __VERIFIER_error print the JSON report only when the mode is unset. The report is serialised under the state lock and written after it, and a stdout error only goes to stderr, so it can no longer drop a verdict or skip the abort. When the state lock is poisoned at the end of a run, the run now ends as develop's UNKNOWN: the NONE already written is removed, map2check_property__unknown is written and the process aborts. The golden tests check that the child's stdout holds only the markers. The C harness checks a file's absence with access(). --- TODO.md | 12 +++- docs/api-deltas.md | 25 +++++--- docs/integration-guide.md | 3 +- docs/semantic-deltas.md | 8 ++- rust/src/ffi.rs | 45 +++++--------- rust/src/ffi_golden_tests.rs | 104 +++++++++++++++++++++++++++++++-- rust/src/nondet/libfuzzer.rs | 4 +- rust/src/output/mod.rs | 28 ++++++++- rust/src/output/termination.rs | 58 ++++++++++++------ tests/ffi_harness.c | 11 +++- 10 files changed, 224 insertions(+), 74 deletions(-) diff --git a/TODO.md b/TODO.md index 969a1cc..97a2cd1 100644 --- a/TODO.md +++ b/TODO.md @@ -163,8 +163,16 @@ e não repetem as tarefas já listadas. núcleo; incluir a matriz completa de símbolos e resolver/documentar a colisão de assinatura de `map2check_check_deref`. Excluir explicitamente símbolos wasm se permanecerem fora do escopo. - -- [ ] **Compatibilizar o canal de resultado legado.** Além do JSON, implementar + Ao passar o endereço da variável-ponteiro em `map2check_free`, fechar a + parte do SD-17 sobre `list_log.csv`: a linha FREE com a variável certa e a + linha que o develop acrescenta antes de um `free` inválido, com um golden + do caminho `map2check_free` que falha (hoje o golden FALSE-FREE usa + `free_resolved_address`; ver `tests/legacy_golden/README.md`). + +- [x] **Compatibilizar o canal de resultado legado.** Feito na Task 2.4 + (#19): `rust/src/output/legacy_files.rs` e golden tests em + `tests/legacy_golden/`. Pendências em SD-17. + Texto original: Além do JSON, implementar a escrita dos arquivos-sentinela `map2check_property_*` nos nomes e formatos aceitos pelo parser atual do Map2Check. Adicionar golden tests para arquivos e JSON e validar propriedade, linha e função contra saídas da biblioteca C diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 7eead9a..a8f6469 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -45,7 +45,7 @@ build with `--features libfuzzer`. |---|---|---|---|---| | `map2check_init` | `void map2check_init(void)` | `map2check_init` | same | Starts a fresh analysis through the `analysis_init` of the selected mode and keeps the options and the mode (#10, SD-14). With a mode set, also removes the `map2check_property*` files left in the current directory (see “Result files”). Map2Check passes call `map2check_init(int)`; see the pass matrix. | | `map2check_reset` | `void map2check_reset(void)` | — | new | Clears containers, result and step counter (#9). Keeps the options set by `map2check_set_memcleanup()` and `map2check_set_null_is_valid()` and the mode set by `map2check_set_mode()`: they are configuration, not analysis state (#10, SD-14). | -| `map2check_success` | `void map2check_success(void)` | `map2check_success` | same | Prints the JSON report (SD-01). With a mode set it is develop's `map2check_success`: writes `NONE`, runs the end-of-program verdict, writes the witness files when enabled, and returns; a violating verdict writes its property and aborts (see “Result files”). | +| `map2check_success` | `void map2check_success(void)` | `map2check_success` | same | Prints the JSON report when the mode is unset (SD-01). With a mode set it prints nothing and is develop's `map2check_success`: writes `NONE`, runs the end-of-program verdict, writes the witness files when enabled, and returns; a violating verdict writes its property and aborts (see “Result files”). | | `set_false_result` | `void set_false_result(int32_t prp, int32_t line_number, const char *function_name)` | `set_false_result` | same | `ViolatedProperty` keeps the C variant order (`caller_lib_result.h:22-31`); the appended `Assert` (8) and `Unknown` (9) are accepted; unknown values are ignored (SD-09). Ignored when a violation is already recorded (SD-03). | | `get_current_property` | `enum ViolatedProperty get_current_property(void)` | `get_current_property` | same | | | `get_current_step` | `uint64_t get_current_step(void)` | `long get_current_step(void)` | changed | Returns `uint64_t` instead of `long`. | @@ -72,7 +72,7 @@ build with `--features libfuzzer`. | `map2check_map_calloc` | `void map2check_map_calloc(const void *ptr_address, int32_t quantity, int32_t size)` | `map2check_map_calloc` | same | Negative sizes are ignored (SD-09). | | `map2check_map_free` | `void map2check_map_free(const char *var_name, const void *ptr_address, uint32_t scope, uint32_t line_number, const char *function_name)` | `map2check_map_free` | same | | | `map2check_map_store_pointer` | `void map2check_map_store_pointer(const void *var_address, const void *value, uint32_t scope, const char *var_name, int32_t line_number, const char *funct_name)` | `map2check_map_store_pointer` | same | | -| `map2check_is_valid_assert` | `void map2check_is_valid_assert(int32_t line_number, const char *function_name, int32_t expression)` | `map2check_is_valid_assert` | changed | A failed assertion reports `ASSERT` (before Task 2.3: `REACHABILITY`) and prints the JSON report. Active in unset and assert modes only (SD-14). | +| `map2check_is_valid_assert` | `void map2check_is_valid_assert(int32_t line_number, const char *function_name, int32_t expression)` | `map2check_is_valid_assert` | changed | A failed assertion reports `ASSERT` (before Task 2.3: `REACHABILITY`) and, with the mode unset, prints the JSON report; with a mode set it ends the run (“Result files”) and prints nothing. Active in unset and assert modes only (SD-14). | | `map2check_binop_add_int` | `void map2check_binop_add_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_add_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | | `map2check_binop_sub_int` | `void map2check_binop_sub_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_sub_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | | `map2check_binop_mul_int` | `void map2check_binop_mul_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_mul_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | @@ -106,7 +106,7 @@ build with `--features libfuzzer`. | `__VERIFIER_nondet_pointer` | `void *__VERIFIER_nondet_pointer(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | | `__VERIFIER_assume` | `void __VERIFIER_assume(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assume(0)` ends the process with `exit(0)` (SD-07). | | `map2check_fuzzer_assume` | `void map2check_fuzzer_assume(int32_t condition)` | `map2check_fuzzer_assume(int)` (`nondet_gen_libfuzzer.h:41`) | same | Feature `libfuzzer`. Same `exit(0)` behaviour (SD-07). | -| `__VERIFIER_error` | `void __VERIFIER_error(void)` | — | new | Feature `libfuzzer`. Prints a REACHABILITY report, then aborts (SD-08). | +| `__VERIFIER_error` | `void __VERIFIER_error(void)` | — | new | Feature `libfuzzer`. Records REACHABILITY, prints the JSON report when the mode is unset, then aborts (SD-08). | | `__VERIFIER_assert` | `void __VERIFIER_assert(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assert(0)` calls `__VERIFIER_error()` (SD-08). | | `next_current_step` | — | `next_current_step()` | discontinued | Use `get_next_step()`; same effect (#13). | | `map2check_save_memtrack_log` | — | `memtracklog.h:44` | discontinued | Covered by the `map2check_map_*` exports; no external caller. | @@ -187,9 +187,15 @@ The Map2Check frontend does not read JSON. It reads the files that the runtime writes into the current directory and learns of a violation from the process aborting (`modules/frontend/utils/tools.cpp`, `caller.cpp`, `counter_example/`). That file channel is the result channel Map2Check -consumes. The JSON report of `print_all_containers_as_json()` and -`map2check_success()` is an additional channel with its own schema -(SD-01), pinned by `rust/src/output/mod.rs::json_report_snapshot` (#14). +consumes. The JSON report of `print_all_containers_as_json()`, +`map2check_success()`, a failing `map2check_is_valid_assert()` and +`__VERIFIER_error()` is an additional channel with its own schema (SD-01), +pinned by `rust/src/output/mod.rs::json_report_snapshot` (#14). It is +printed only when the mode is unset: with a mode set no export writes to +stdout, as in develop, so the analysed program's output stays clean. The +report is serialised under the state lock and printed after the lock is +released, and a stdout error is only reported on stderr, so stdout can +never change a verdict. The channel is tied to the mode. When `map2check_set_mode()` was called (a Map2Check-instrumented program): @@ -201,7 +207,12 @@ The channel is tied to the mode. When `map2check_set_mode()` was called `map2check_error` → `map2check_exit` (`Map2CheckFunctions.c:72-102`); - `map2check_success()` writes `NONE`, runs the end-of-program verdict and, if it holds, writes the witness files (witness on) and returns; - otherwise it ends the run as a violation. + otherwise it ends the run as a violation; +- if the analysis state cannot be read at the end (its lock was poisoned + by an earlier panic), the verdict is unknown: the property files already + written are removed, `map2check_property__unknown` is written (plus + `map2check_checked_error` with witness on) and the process aborts, as + develop's UNKNOWN path does. When the mode is unset (direct API users, `tests/ffi_harness.c`) nothing is written and nothing ends the process: violations are only recorded. diff --git a/docs/integration-guide.md b/docs/integration-guide.md index 3c2481f..0068676 100644 --- a/docs/integration-guide.md +++ b/docs/integration-guide.md @@ -88,7 +88,8 @@ place (SD-03). Only `map2check_init()`, `map2check_reset()` and ## JSON report `print_all_containers_as_json()` prints one pretty-printed JSON object on -stdout. Whitespace is not part of the contract; parse it. The examples +stdout when no analysis mode is set (with a mode set, a Map2Check run, it +prints nothing; see below). Whitespace is not part of the contract; parse it. The examples below are checked against the real serializer by `tests/compat_docs.rs::integration_guide_json_examples_match_the_serializer`. diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 249f0ca..759c5d7 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -35,7 +35,7 @@ entry and its tests in the same change. ### SD-01 — JSON report shape **Classification:** intentional (#14) -**Tests:** `tests/compat_docs.rs::integration_guide_json_examples_match_the_serializer`, `rust/src/output/mod.rs::output_is_valid_json`, `rust/src/output/mod.rs::json_report_snapshot` +**Tests:** `tests/compat_docs.rs::integration_guide_json_examples_match_the_serializer`, `rust/src/output/mod.rs::output_is_valid_json`, `rust/src/output/mod.rs::json_report_snapshot`, `rust/src/ffi_golden_tests.rs::assert_violation_with_mode_set_writes_files_aborts_and_prints_no_json` **Rust:** `print_all_containers_as_json()`, `map2check_success()` and a failing `map2check_is_valid_assert()` print one pretty-printed JSON object @@ -56,7 +56,8 @@ part of the schema. Whitespace alone is not a semantic difference. files that the library writes when an analysis mode is set (#19, see “Result files” in [`api-deltas.md`](api-deltas.md#result-files)). The JSON is an additional channel with its own schema, kept as it is and pinned by -`json_report_snapshot`. +`json_report_snapshot`. It is printed only when no analysis mode is set; a +Map2Check run (mode set) prints nothing on stdout, as develop. **Reproduce:** @@ -258,7 +259,8 @@ __VERIFIER_assume(0); /* process exits with status 0 */ **Tests:** `rust/src/nondet/libfuzzer.rs::verifier_error_aborts_and_reports_reachability`, `rust/src/nondet/libfuzzer.rs::verifier_assert_false_aborts_and_reports_reachability` **Rust:** `__VERIFIER_error()` and `__VERIFIER_assert(0)` record -`REACHABILITY`, print the JSON report and call `abort()`. LibFuzzer treats +`REACHABILITY`, print the JSON report (only when no analysis mode is +set, SD-01) and call `abort()`. LibFuzzer treats the abort as a crash. **Legacy (embedded C library):** a violation ends in `abort()` through diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index e0d81c0..efbd0b1 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -163,26 +163,20 @@ pub extern "C" fn map2check_reset() { // ── Result management ───────────────────────────────────────────────────────── -/// End of the analysed program. Prints the JSON report. With an analysis -/// mode set it is develop's `map2check_success()` +/// End of the analysed program. With the mode unset, prints the JSON +/// report. With an analysis mode set it prints nothing and is develop's +/// `map2check_success()` /// (`Map2CheckFunctions.c:67-70`, `:89-102`): writes `NONE` to /// `map2check_property`, runs the end-of-program verdict of the mode, writes /// the witness files when `map2check_enable_witness()` was called, and /// returns; when the verdict is a violation it writes that property instead -/// and aborts. The JSON is printed after the verdict. +/// and aborts. #[no_mangle] pub extern "C" fn map2check_success() { ffi_guard( || { - let print = |s: &state::AnalysisState| { - if let Err(e) = output::print_json(s) { - log_error(&e); - } - }; - if !termination::on_success(print) { - if let Err(e) = state::with_state(|s| output::print_json(s)) { - log_error(&e); - } + if !termination::on_success() { + output::print_report_unless_mode_set(); } }, (), @@ -239,17 +233,12 @@ pub extern "C" fn get_next_step() -> u64 { ffi_guard(|| state::with_state(|s| Ok(s.next_step())).unwrap_or(0), 0) } -/// Prints the full JSON report to stdout. +/// Prints the full JSON report to stdout when no analysis mode is set. With +/// a mode set (a Map2Check run) it prints nothing: the result goes to the +/// files, and the analysed program's stdout stays clean. #[no_mangle] pub extern "C" fn print_all_containers_as_json() { - ffi_guard( - || { - if let Err(e) = state::with_state(|s| output::print_json(s)) { - log_error(&e); - } - }, - (), - ); + ffi_guard(output::print_report_unless_mode_set, ()); } /// Resets violation metadata without clearing tracking containers. @@ -951,16 +940,10 @@ pub unsafe extern "C" fn map2check_is_valid_assert( let checker = AssertChecker { expression }; match checker.check(&ctx) { Ok(crate::analysismode::VccOutcome::Violated { property }) => { - match state::with_state(|s| { - let first = s.result.ok; - s.set_false(property, line_number as u32, fname); - output::print_json(s)?; - Ok(first) - }) { - Ok(true) => termination::on_violation(false), - Ok(false) => {} - Err(e) => log_error(&e), - } + // With a mode set the run ends inside; unset, the JSON + // report follows the recorded violation. + record_violation(property, line_number as u32, fname, false); + output::print_report_unless_mode_set(); } Ok(crate::analysismode::VccOutcome::Safe) => {} Err(e) => log_error(&e), diff --git a/rust/src/ffi_golden_tests.rs b/rust/src/ffi_golden_tests.rs index 7988ae2..80aa46a 100644 --- a/rust/src/ffi_golden_tests.rs +++ b/rust/src/ffi_golden_tests.rs @@ -23,6 +23,7 @@ use crate::analysismode::mode::AnalysisMode; const CHILD_ENV: &str = "MAP2CHECK_GOLDEN_CHILD"; const STARTED: &str = "map2check-golden-child-started"; +const RETURNED: &str = "map2check-golden-child-returned"; /// Fixed addresses of `tests/legacy_golden/drivers/golden.h`. const ARENA: usize = 0x1000_0000; @@ -67,6 +68,29 @@ fn files_in(dir: &Path) -> BTreeSet { .collect() } +/// With a mode set the library writes nothing to stdout (no JSON report): +/// after the start marker the child prints nothing until it aborts, or +/// until the `RETURNED` marker printed right after `map2check_success()`. +fn assert_quiet_stdout(stdout: &str, aborted: bool) { + assert!( + !stdout.contains("map2check_log"), + "JSON no stdout:\n{stdout}" + ); + let after = stdout + .split_once(STARTED) + .map(|(_, rest)| rest) + .expect("marcador de início"); + let produced = if aborted { + after + } else { + after + .split_once(RETURNED) + .map(|(before, _)| before) + .expect("marcador de retorno") + }; + assert_eq!(produced, "\n", "saída inesperada no stdout:\n{stdout}"); +} + /// Runs the `` scenario and compares its files and its end with /// `tests/legacy_golden//`. fn assert_matches_golden(tag: &str, child: &str) { @@ -93,6 +117,8 @@ fn assert_matches_golden(tag: &str, child: &str) { other => panic!("{tag}: .exit_status inesperado `{other}`"), } + assert_quiet_stdout(&stdout, status.trim() == "134"); + assert_eq!( files_in(dir.path()), files_in(&golden), @@ -235,6 +261,7 @@ fn none_child() { memory(); free_p(); map2check_success(); + println!("{RETURNED}"); } } @@ -344,7 +371,7 @@ fn success_without_witness_child() { memory(); free_p(); map2check_success(); - println!("map2check-golden-child-returned"); + println!("{RETURNED}"); } } @@ -353,7 +380,7 @@ fn without_witness_success_writes_none_and_returns() { let dir = tempfile::tempdir().unwrap(); let output = run_child("success_without_witness_child", dir.path()); assert_eq!(output.status.code(), Some(0)); - assert!(String::from_utf8_lossy(&output.stdout).contains("map2check-golden-child-returned")); + assert!(String::from_utf8_lossy(&output.stdout).contains(RETURNED)); assert_eq!( files_in(dir.path()), BTreeSet::from(["map2check_property".to_owned()]) @@ -403,7 +430,7 @@ fn unset_mode_records_without_files_or_abort_child() { crate::caller::ViolatedProperty::Overflow ); map2check_success(); - println!("map2check-golden-child-returned"); + println!("{RETURNED}"); } } @@ -416,9 +443,78 @@ fn unset_mode_records_without_files_or_abort() { dir.path(), ); assert_eq!(output.status.code(), Some(0)); - assert!(String::from_utf8_lossy(&output.stdout).contains("map2check-golden-child-returned")); + assert!(String::from_utf8_lossy(&output.stdout).contains(RETURNED)); assert_eq!( files_in(dir.path()), BTreeSet::from(["map2check_property_free".to_owned()]) ); } + +// ── No JSON on stdout with a mode set; a poisoned state is UNKNOWN ───────── + +#[test] +fn assert_violation_without_witness_child() { + if is_child() { + println!("{STARTED}"); + map2check_set_mode(AnalysisMode::Assert as i32); + map2check_init(); + print_all_containers_as_json(); + // SAFETY: C-string literal is valid. + unsafe { map2check_is_valid_assert(21, c"main".as_ptr(), 0) }; + println!("map2check-golden-child-survived"); + } +} + +#[test] +fn assert_violation_with_mode_set_writes_files_aborts_and_prints_no_json() { + let dir = tempfile::tempdir().unwrap(); + let output = run_child("assert_violation_without_witness_child", dir.path()); + let stdout = String::from_utf8_lossy(&output.stdout); + assert_eq!(output.status.signal(), Some(6), "{stdout}"); + assert_quiet_stdout(&stdout, true); + assert_eq!( + files_in(dir.path()), + BTreeSet::from([ + "map2check_property".to_owned(), + "map2check_property_assert".to_owned() + ]) + ); + assert_eq!( + std::fs::read(dir.path().join("map2check_property_assert")).unwrap(), + b"ASSERT\nLine: 21\nFunction: main\n" + ); +} + +#[test] +fn poisoned_state_at_success_child() { + if is_child() { + println!("{STARTED}"); + map2check_set_mode(AnalysisMode::Memtrack as i32); + map2check_enable_witness(); + map2check_init(); + let _ = std::thread::spawn(|| { + let _held = crate::state::get().unwrap().lock().unwrap(); + panic!("poison the analysis state on purpose"); + }) + .join(); + map2check_success(); + println!("map2check-golden-child-survived"); + } +} + +#[test] +fn poisoned_state_at_success_reports_unknown_and_aborts() { + let dir = tempfile::tempdir().unwrap(); + let output = run_child("poisoned_state_at_success_child", dir.path()); + let stdout = String::from_utf8_lossy(&output.stdout); + assert_eq!(output.status.signal(), Some(6), "{stdout}"); + assert!(!stdout.contains("survived"), "{stdout}"); + // No NONE left behind: develop's UNKNOWN (SD-16) plus the confirmation. + assert_eq!( + files_in(dir.path()), + BTreeSet::from([ + "map2check_property__unknown".to_owned(), + "map2check_checked_error".to_owned() + ]) + ); +} diff --git a/rust/src/nondet/libfuzzer.rs b/rust/src/nondet/libfuzzer.rs index 0816ef9..2593b72 100644 --- a/rust/src/nondet/libfuzzer.rs +++ b/rust/src/nondet/libfuzzer.rs @@ -226,10 +226,12 @@ pub extern "C" fn __VERIFIER_error() { 0, "__VERIFIER_error", ); - crate::output::print_json(s) + Ok(()) }) { eprintln!("[map2check] error: {e}"); } + // Out of the lock, and only without a mode (#19 ruling). + crate::output::print_report_unless_mode_set(); }, (), ); diff --git a/rust/src/output/mod.rs b/rust/src/output/mod.rs index efa0276..3476f74 100644 --- a/rust/src/output/mod.rs +++ b/rust/src/output/mod.rs @@ -79,13 +79,35 @@ pub fn to_json(state: &AnalysisState) -> Result { serde_json::to_string_pretty(&log).map_err(Map2CheckError::from) } -/// Print the JSON report to stdout, as the C library did. +/// Print the JSON report to stdout, as the C library did. A failed write +/// (closed stdout, EPIPE) is reported on stderr instead of panicking. pub fn print_json(state: &AnalysisState) -> Result<(), Map2CheckError> { - let json = to_json(state)?; - println!("{json}"); + write_report(&to_json(state)?); Ok(()) } +fn write_report(json: &str) { + use std::io::Write as _; + if let Err(e) = writeln!(std::io::stdout().lock(), "{json}") { + eprintln!("[map2check] cannot print the JSON report: {e}"); + } +} + +/// The JSON report channel of the exports (#14). Prints only when no +/// analysis mode is set: a Map2Check run reports through the result files +/// and keeps the analysed program's stdout clean, as develop does. The +/// report is serialised under the state lock and written after the lock is +/// released, so stdout can never affect the state or the verdict. +pub fn print_report_unless_mode_set() { + if crate::state::config().mode.is_some() { + return; + } + match crate::state::with_state(|s| to_json(s)) { + Ok(json) => write_report(&json), + Err(e) => eprintln!("[map2check] error: {e}"), + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/rust/src/output/termination.rs b/rust/src/output/termination.rs index 9534614..f9f4b99 100644 --- a/rust/src/output/termination.rs +++ b/rust/src/output/termination.rs @@ -90,28 +90,52 @@ fn write_aux_files(dir: &Path, config: Config, s: &AnalysisState, end_of_program /// `analysis_destroy`, then, on a violation, the confirmation file and the /// abort. Returns only when there is no violation. fn exit(dir: &Path, config: Config, end_of_program: bool) { - let failed = state::with_state(|s| { + let failed = match state::with_state(|s| { if config.witness { write_aux_files(dir, config, s, end_of_program); } mode::analysis_destroy(config.mode, s); Ok(!s.result.ok) - }) - .unwrap_or_else(|e| { - eprintln!("[map2check] error: {e}"); - true - }); + }) { + Ok(failed) => failed, + Err(e) => unknown_and_abort(dir, config, &e), + }; if failed { - if config.witness { - report( - legacy_files::CHECKED_ERROR_FILE, - legacy_files::write_checked_error(dir), - ); - } + write_checked_error(dir, config); std::process::abort(); } } +fn write_checked_error(dir: &Path, config: Config) { + if config.witness { + report( + legacy_files::CHECKED_ERROR_FILE, + legacy_files::write_checked_error(dir), + ); + } +} + +/// The analysis state cannot be read (its lock was poisoned by an earlier +/// panic), so the verdict is not known. Ends the run the way develop ends +/// an undecided one, `write_property_unknown` + `map2check_error` +/// (`AnalysisModeMemtrack.c:305-311`): any property file already written +/// (`NONE` from `map2check_success`) is removed, `map2check_property__unknown` +/// is written, then `map2check_checked_error` with witness on, and the +/// process aborts. The frontend reads UNKNOWN because `map2check_property` +/// is missing (`tools.cpp:98-103`). No log files: their data is unreadable. +fn unknown_and_abort(dir: &Path, config: Config, error: &crate::error::Map2CheckError) -> ! { + eprintln!("[map2check] error: {error}; the verdict is UNKNOWN"); + legacy_files::remove_stale_property_files(dir); + let unknown = AnalysisResult { + ok: false, + property: crate::caller::ViolatedProperty::Unknown, + ..AnalysisResult::default() + }; + write_properties(dir, &unknown); + write_checked_error(dir, config); + std::process::abort(); +} + fn write_properties(dir: &Path, result: &AnalysisResult) { report( "the property files", @@ -131,7 +155,7 @@ pub(crate) fn on_violation(end_of_program: bool) { }; match state::with_state(|s| Ok(s.result.clone())) { Ok(result) => write_properties(&dir, &result), - Err(e) => eprintln!("[map2check] error: {e}"), + Err(e) => unknown_and_abort(&dir, config, &e), } exit(&dir, config, end_of_program); // `exit` returns only without a violation, which a recorded violation @@ -143,7 +167,7 @@ pub(crate) fn on_violation(end_of_program: bool) { /// mode set, writes `NONE`, runs the end-of-program verdict, ends the run, /// and returns when the program is correct. Returns whether the file /// channel handled the call. -pub(crate) fn on_success(after_verdict: impl FnOnce(&AnalysisState)) -> bool { +pub(crate) fn on_success() -> bool { let config = state::config(); let Some(dir) = channel(config) else { return false; @@ -156,7 +180,6 @@ pub(crate) fn on_success(after_verdict: impl FnOnce(&AnalysisState)) -> bool { s.set_false(property, 0, ""); } } - after_verdict(s); Ok((end_of_program, s.result.clone())) }); let end_of_program = match verdict { @@ -166,10 +189,7 @@ pub(crate) fn on_success(after_verdict: impl FnOnce(&AnalysisState)) -> bool { } end_of_program } - Err(e) => { - eprintln!("[map2check] error: {e}"); - false - } + Err(e) => unknown_and_abort(&dir, config, &e), }; exit(&dir, config, end_of_program); true diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index 340ebd0..958d3e0 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -398,7 +398,8 @@ static void remove_results(const char *dir) { /* Runs `scenario` in a child. `aborts`: whether it must end with SIGABRT * (otherwise exit 0). `property`: expected map2check_property content. - * `extra`/`extra_content`: one more file to check, or NULL. */ + * `extra`: one more file to check, or NULL; `extra_content`: its expected + * content, or NULL when the file must not exist. */ static int run_mode_scenario(scenario_fn scenario, int aborts, const char *property, const char *extra, const char *extra_content) { @@ -428,7 +429,11 @@ static int run_mode_scenario(scenario_fn scenario, int aborts, char buf[256]; read_result(dir, "map2check_property", buf, sizeof(buf)); failed |= strcmp(buf, property) != 0; - if (extra != NULL) { + if (extra != NULL && extra_content == NULL) { + char path[512]; + snprintf(path, sizeof(path), "%s/%s", dir, extra); + failed |= access(path, F_OK) == 0; + } else if (extra != NULL) { read_result(dir, extra, buf, sizeof(buf)); failed |= strcmp(buf, extra_content) != 0; } @@ -513,7 +518,7 @@ static int test_analysis_modes(void) { } if (run_mode_scenario(scenario_memtrack_deref, 1, "FALSE-DEREF\nLine: 1\nFunction: main\n", - "map2check_checked_error", "")) { + "map2check_checked_error", NULL)) { return 2; } if (run_mode_scenario(scenario_memtrack_deref_witness, 1, From 9d955b0a191edb1e6fdba1e638a060ba70f88f54 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 22:36:14 -0400 Subject: [PATCH 26/63] feat: check signed binops at the operation's width and report division by zero (#13) Add map2check_binop_{add,sub,mul,div,rem}_signed(a, b, bits, line, fn), the core targets of develop's map2check_binop_{add,sub,mul,sdiv,srem} (AnalysisModeOverflow.c:110-276): 64-bit predicates, then the range of the operation's width; MIN / -1 and MIN % -1 overflow; a zero divisor reports the new ViolatedProperty::DivByZero (10), written as FALSE-DIVBYZERO to map2check_property only (PropertyGenerator.c:98-102). Gated like the other binops. Golden DIVBYZERO captured from develop. --- docs/api-deltas.md | 13 +- rust/src/analysismode/overflow.rs | 67 ++++++++ rust/src/caller/mod.rs | 7 + rust/src/ffi.rs | 140 ++++++++++++++++ rust/src/ffi_golden_tests.rs | 15 ++ rust/src/ffi_tests.rs | 150 ++++++++++++++++++ rust/src/output/legacy_files.rs | 18 +++ tests/ffi_harness.c | 17 ++ tests/legacy_golden/DIVBYZERO/.exit_status | 1 + tests/legacy_golden/DIVBYZERO/klee_log.csv | 5 + .../DIVBYZERO/map2check_checked_error | 1 + .../DIVBYZERO/map2check_property | 3 + tests/legacy_golden/DIVBYZERO/track_bb_log.st | 2 + tests/legacy_golden/README.md | 1 + tests/legacy_golden/drivers/DIVBYZERO.c | 13 ++ tests/legacy_golden/drivers/build_goldens.sh | 3 +- tests/legacy_golden/drivers/golden.h | 2 + 17 files changed, 453 insertions(+), 5 deletions(-) create mode 100644 tests/legacy_golden/DIVBYZERO/.exit_status create mode 100644 tests/legacy_golden/DIVBYZERO/klee_log.csv create mode 100644 tests/legacy_golden/DIVBYZERO/map2check_checked_error create mode 100644 tests/legacy_golden/DIVBYZERO/map2check_property create mode 100644 tests/legacy_golden/DIVBYZERO/track_bb_log.st create mode 100644 tests/legacy_golden/drivers/DIVBYZERO.c diff --git a/docs/api-deltas.md b/docs/api-deltas.md index a8f6469..1d28a27 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -46,7 +46,7 @@ build with `--features libfuzzer`. | `map2check_init` | `void map2check_init(void)` | `map2check_init` | same | Starts a fresh analysis through the `analysis_init` of the selected mode and keeps the options and the mode (#10, SD-14). With a mode set, also removes the `map2check_property*` files left in the current directory (see “Result files”). Map2Check passes call `map2check_init(int)`; see the pass matrix. | | `map2check_reset` | `void map2check_reset(void)` | — | new | Clears containers, result and step counter (#9). Keeps the options set by `map2check_set_memcleanup()` and `map2check_set_null_is_valid()` and the mode set by `map2check_set_mode()`: they are configuration, not analysis state (#10, SD-14). | | `map2check_success` | `void map2check_success(void)` | `map2check_success` | same | Prints the JSON report when the mode is unset (SD-01). With a mode set it prints nothing and is develop's `map2check_success`: writes `NONE`, runs the end-of-program verdict, writes the witness files when enabled, and returns; a violating verdict writes its property and aborts (see “Result files”). | -| `set_false_result` | `void set_false_result(int32_t prp, int32_t line_number, const char *function_name)` | `set_false_result` | same | `ViolatedProperty` keeps the C variant order (`caller_lib_result.h:22-31`); the appended `Assert` (8) and `Unknown` (9) are accepted; unknown values are ignored (SD-09). Ignored when a violation is already recorded (SD-03). | +| `set_false_result` | `void set_false_result(int32_t prp, int32_t line_number, const char *function_name)` | `set_false_result` | same | `ViolatedProperty` keeps the C variant order (`caller_lib_result.h:22-31`); the appended `Assert` (8), `Unknown` (9) and `DivByZero` (10) are accepted; unknown values are ignored (SD-09). Ignored when a violation is already recorded (SD-03). | | `get_current_property` | `enum ViolatedProperty get_current_property(void)` | `get_current_property` | same | | | `get_current_step` | `uint64_t get_current_step(void)` | `long get_current_step(void)` | changed | Returns `uint64_t` instead of `long`. | | `get_next_step` | `uint64_t get_next_step(void)` | `get_next_step()` / `next_current_step()` | same | Advances and returns the step counter. `next_current_step` is not exported; see its row. | @@ -85,6 +85,11 @@ build with `--features libfuzzer`. | `map2check_binop_mul_unsigned` | `void map2check_binop_mul_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_mul_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | | `map2check_binop_div_unsigned` | `void map2check_binop_div_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_div_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | | `map2check_binop_shr_unsigned` | `void map2check_binop_shr_unsigned(uint32_t param1, uint32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_shr_unsigned` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | +| `map2check_binop_add_signed` | `void map2check_binop_add_signed(int64_t a, int64_t b, uint32_t bits, uint32_t line, const char *function_name)` | `map2check_binop_add` (develop) | changed | Operands sign-extended to 64 bits, `bits` the width of the result type (0 or above 64 means 64), as the pass sends them; reports `OVERFLOW` when the exact sum leaves the range of `bits` (`AnalysisModeOverflow.c:110-276`). No `scope` argument. Records and continues with the mode unset; ends the run with a mode set (SD-03). Active in unset and overflow modes only (SD-14). | +| `map2check_binop_sub_signed` | `void map2check_binop_sub_signed(int64_t a, int64_t b, uint32_t bits, uint32_t line, const char *function_name)` | `map2check_binop_sub` (develop) | changed | Operands sign-extended to 64 bits, `bits` the width of the result type (0 or above 64 means 64), as the pass sends them; reports `OVERFLOW` when the exact difference leaves the range of `bits` (`AnalysisModeOverflow.c:110-276`). No `scope` argument. Records and continues with the mode unset; ends the run with a mode set (SD-03). Active in unset and overflow modes only (SD-14). | +| `map2check_binop_mul_signed` | `void map2check_binop_mul_signed(int64_t a, int64_t b, uint32_t bits, uint32_t line, const char *function_name)` | `map2check_binop_mul` (develop) | changed | Operands sign-extended to 64 bits, `bits` the width of the result type (0 or above 64 means 64), as the pass sends them; reports `OVERFLOW` when the exact product leaves the range of `bits` (`AnalysisModeOverflow.c:110-276`). No `scope` argument. Records and continues with the mode unset; ends the run with a mode set (SD-03). Active in unset and overflow modes only (SD-14). | +| `map2check_binop_div_signed` | `void map2check_binop_div_signed(int64_t a, int64_t b, uint32_t bits, uint32_t line, const char *function_name)` | `map2check_binop_sdiv` (develop) | changed | Operands sign-extended to 64 bits, `bits` the width of the result type (0 or above 64 means 64), as the pass sends them; reports `DIVBYZERO` (`ViolatedProperty` 10, develop `FALSE-DIVBYZERO`) when `b` is 0, `OVERFLOW` for MIN / -1 of `bits` (`AnalysisModeOverflow.c:110-276`). No `scope` argument. Records and continues with the mode unset; ends the run with a mode set (SD-03). Active in unset and overflow modes only (SD-14). | +| `map2check_binop_rem_signed` | `void map2check_binop_rem_signed(int64_t a, int64_t b, uint32_t bits, uint32_t line, const char *function_name)` | `map2check_binop_srem` (develop) | changed | Operands sign-extended to 64 bits, `bits` the width of the result type (0 or above 64 means 64), as the pass sends them; reports `DIVBYZERO` when `b` is 0, `OVERFLOW` for MIN % -1 of `bits` (`AnalysisModeOverflow.c:110-276`). No `scope` argument. Records and continues with the mode unset; ends the run with a mode set (SD-03). Active in unset and overflow modes only (SD-14). | | `map2check_check_load` | `void map2check_check_load(const void *ptr, int32_t line, uint32_t scope, int32_t size, const char *function_name)` | `map2check_check_load` | changed | Address-validity rule rewritten (plan R1; differential check in #15); since #11 a heap block is valid only while its latest alloc/free event is an allocation, as in `develop` (`AllocationLog.c:127-148`). The first violation is final; a later check never clears or replaces it (SD-03); NULL handling in SD-05. Active in unset and memtrack modes only (SD-14). | | `map2check_check_free` | `void map2check_check_free(const char *_name, const void *ptr, uint32_t _scope, uint32_t line, const char *function_name)` | `map2check_check_free` | same | `_name` and `_scope` are ignored. Untracked address is a violation (SD-06). Active in unset and memtrack modes only (SD-14). | | `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; records the release when the check passes, unlike `map2check_check_free`; untracked address is a violation (SD-06); see also SD-13. The check is active in unset and memtrack modes only; when inactive the release is still recorded (SD-14). | @@ -178,8 +183,8 @@ per-mode `match` is in `rust/src/analysismode/mode.rs`. Tracking calls The memcleanup option only changes the tag of the unset/memtrack leak verdict; it does not apply the memcleanup-mode reachability rule. `map2check_reach_target` and `set_false_result` record in every mode. -`ViolatedProperty` gained `Assert` (8) and `Unknown` (9), appended after -`None`; `Reachability` is the target-reached property. +`ViolatedProperty` gained `Assert` (8), `Unknown` (9) and `DivByZero` (10), +appended after `None`; `Reachability` is the target-reached property. ## Result files @@ -222,7 +227,7 @@ In memtrack and memcleanup modes `map2check_map_malloc` and | File | Develop writer | Rust writer (`rust/src/output/`) | When | |---|---|---|---| -| `map2check_property` | `PropertyGenerator.c:50-106` | `legacy_files::write_property_files` | mode set: on the first violation (every property but `UNKNOWN`) and `NONE` on `map2check_success` | +| `map2check_property` | `PropertyGenerator.c:50-106` | `legacy_files::write_property_files` | mode set: on the first violation (every property but `UNKNOWN`; `DIVBYZERO` writes `FALSE-DIVBYZERO` here only, as `:98-102`) and `NONE` on `map2check_success` | | `map2check_property_{free,deref,overflow,assert,memtrack,memcleanup}` | `PropertyGenerator.c:24-48`, `:108-130` | `legacy_files::write_property_files` | mode set: with `map2check_property` for that property | | `map2check_property__unknown` | `PropertyGenerator.c:16`, `:24-28` | `legacy_files::write_property_files` | mode set: `UNKNOWN` (double underscore kept, SD-16) | | `track_bb_log.st` | `TrackBBLog.c:35-44` | `legacy_files::write_trackbb_log` | mode set, witness on: end of run | diff --git a/rust/src/analysismode/overflow.rs b/rust/src/analysismode/overflow.rs index acb1d15..fc00eed 100644 --- a/rust/src/analysismode/overflow.rs +++ b/rust/src/analysismode/overflow.rs @@ -187,6 +187,73 @@ impl VccChecker for MulI64 { } } +// ── Signed binops at the operation's width (AnalysisModeOverflow.c) ────────── + +/// The signed operation a [`SignedBinop`] checks. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum SignedOp { + Add, + Sub, + Mul, + Div, + Rem, +} + +/// One signed binop of the Map2Check `develop` C runtime +/// (`map2check_binop_{add,sub,mul,sdiv,srem}`, `AnalysisModeOverflow.c:197-276`): +/// both operands sign-extended to 64 bits, `bits` the width of the +/// operation's result type. +pub struct SignedBinop { + pub op: SignedOp, + pub lhs: i64, + pub rhs: i64, + pub bits: u32, +} + +/// `signed_min_for_width` / `signed_max_for_width` (`:111-124`): a width of +/// 0 or 64 and more means 64 bits. +fn signed_bounds(bits: u32) -> (i64, i64) { + if bits == 0 || bits >= 64 { + (i64::MIN, i64::MAX) + } else { + (-(1i64 << (bits - 1)), (1i64 << (bits - 1)) - 1) + } +} + +impl SignedBinop { + /// The develop verdict: `None` when the operation is safe. + pub fn violation(&self) -> Option { + let (min, max) = signed_bounds(self.bits); + let (a, b) = (self.lhs, self.rhs); + // `:126-134` after the 64-bit predicates of `:141-183`: the exact + // result must exist in 64 bits and fit the width. + let fits = |exact: Option| exact.is_some_and(|v| (min..=max).contains(&v)); + let safe = match self.op { + SignedOp::Add => fits(a.checked_add(b)), + SignedOp::Sub => fits(a.checked_sub(b)), + SignedOp::Mul => fits(a.checked_mul(b)), + // `:250-276`: a zero divisor is FALSE_DIVBYZERO; only MIN / -1 + // of the width overflows, and MIN % -1 is undefined in C. + SignedOp::Div | SignedOp::Rem => { + if b == 0 { + return Some(ViolatedProperty::DivByZero); + } + !(a == min && b == -1) + } + }; + (!safe).then_some(ViolatedProperty::Overflow) + } +} + +impl VccChecker for SignedBinop { + fn check(&self, _ctx: &VccContext) -> Result { + Ok(match self.violation() { + Some(property) => VccOutcome::Violated { property }, + None => VccOutcome::Safe, + }) + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/rust/src/caller/mod.rs b/rust/src/caller/mod.rs index df7a27f..65d5c4b 100644 --- a/rust/src/caller/mod.rs +++ b/rust/src/caller/mod.rs @@ -22,6 +22,10 @@ pub enum ViolatedProperty { Assert, /// The analysis could not decide (`UNKNOWN` in `PropertyGenerator.c`). Unknown, + /// Signed division or remainder by zero (`FALSE_DIVBYZERO`, + /// `Map2CheckTypes.h:78`), reported by `map2check_binop_div_signed` and + /// `map2check_binop_rem_signed`. + DivByZero, } impl ViolatedProperty { @@ -37,6 +41,7 @@ impl ViolatedProperty { Self::None => "NONE", Self::Assert => "ASSERT", Self::Unknown => "UNKNOWN", + Self::DivByZero => "DIVBYZERO", } } } @@ -87,6 +92,7 @@ mod tests { assert_eq!(ViolatedProperty::None as i32, 7); assert_eq!(ViolatedProperty::Assert as i32, 8); assert_eq!(ViolatedProperty::Unknown as i32, 9); + assert_eq!(ViolatedProperty::DivByZero as i32, 10); } #[test] @@ -105,6 +111,7 @@ mod tests { (ViolatedProperty::None, "NONE"), (ViolatedProperty::Assert, "ASSERT"), (ViolatedProperty::Unknown, "UNKNOWN"), + (ViolatedProperty::DivByZero, "DIVBYZERO"), ]; for (property, name) in cases { assert_eq!(property.as_str(), name); diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index efbd0b1..d570733 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -25,6 +25,8 @@ use crate::analysismode::overflow::NegI32; use crate::analysismode::overflow::ShlI32; use crate::analysismode::overflow::ShrI32; use crate::analysismode::overflow::ShrU32; +use crate::analysismode::overflow::SignedBinop; +use crate::analysismode::overflow::SignedOp; use crate::analysismode::overflow::SubI32; use crate::analysismode::overflow::SubU32; use crate::analysismode::VccChecker; @@ -110,6 +112,7 @@ fn violated_property_from_raw(raw: i32) -> Option { 7 => Some(ViolatedProperty::None), 8 => Some(ViolatedProperty::Assert), 9 => Some(ViolatedProperty::Unknown), + 10 => Some(ViolatedProperty::DivByZero), _ => None, } } @@ -1063,6 +1066,143 @@ overflow_ffi_shift!(map2check_binop_shl_int, ShlI32, i32); overflow_ffi_shift!(map2check_binop_shr_int, ShrI32, i32); overflow_ffi_shift!(map2check_binop_shr_unsigned, ShrU32, u32); +/// Shared body of the `map2check_binop_*_signed` exports. `a` and `b` are +/// the operands sign-extended to 64 bits and `bits` is the width of the +/// operation's result type, as the Map2Check overflow pass sends them +/// (`OverflowPass.cpp:418-440`); a width of 0 or above 64 means 64. The +/// checks are those of develop's `AnalysisModeOverflow.c:110-276`. Active +/// in unset and overflow modes only, like the other binops. +/// +/// # Safety +/// `function_name` must be null or a valid null-terminated C string. +unsafe fn signed_binop( + op: SignedOp, + a: i64, + b: i64, + bits: u32, + line: u32, + function_name: *const std::os::raw::c_char, +) { + if !check_active(Check::Overflow) { + return; + } + // SAFETY: caller guarantees function_name is valid or null. + let fname = unsafe { c_str(function_name) }; + let ctx = VccContext::new(line, 0, fname); + let checker = SignedBinop { + op, + lhs: a, + rhs: b, + bits, + }; + run_vcc_check(checker, &ctx, line, fname); +} + +/// Signed `a + b`: reports `OVERFLOW` when the sum leaves the range of +/// `bits` (develop `map2check_binop_add`). +/// See [`signed_binop`] for the operands, the width and the mode gating. +/// +/// # Safety +/// `function_name` must be null or a valid null-terminated C string. +#[no_mangle] +pub unsafe extern "C" fn map2check_binop_add_signed( + a: i64, + b: i64, + bits: u32, + line: u32, + function_name: *const std::os::raw::c_char, +) { + ffi_guard( + // SAFETY: forwarded caller contract. + || unsafe { signed_binop(SignedOp::Add, a, b, bits, line, function_name) }, + (), + ); +} + +/// Signed `a - b`: reports `OVERFLOW` when the difference leaves the range +/// of `bits` (develop `map2check_binop_sub`). +/// See [`signed_binop`] for the operands, the width and the mode gating. +/// +/// # Safety +/// `function_name` must be null or a valid null-terminated C string. +#[no_mangle] +pub unsafe extern "C" fn map2check_binop_sub_signed( + a: i64, + b: i64, + bits: u32, + line: u32, + function_name: *const std::os::raw::c_char, +) { + ffi_guard( + // SAFETY: forwarded caller contract. + || unsafe { signed_binop(SignedOp::Sub, a, b, bits, line, function_name) }, + (), + ); +} + +/// Signed `a * b`: reports `OVERFLOW` when the product leaves the range of +/// `bits` (develop `map2check_binop_mul`). +/// See [`signed_binop`] for the operands, the width and the mode gating. +/// +/// # Safety +/// `function_name` must be null or a valid null-terminated C string. +#[no_mangle] +pub unsafe extern "C" fn map2check_binop_mul_signed( + a: i64, + b: i64, + bits: u32, + line: u32, + function_name: *const std::os::raw::c_char, +) { + ffi_guard( + // SAFETY: forwarded caller contract. + || unsafe { signed_binop(SignedOp::Mul, a, b, bits, line, function_name) }, + (), + ); +} + +/// Signed `a / b`: reports `DIVBYZERO` when `b` is 0 and `OVERFLOW` for +/// MIN / -1 of `bits` (develop `map2check_binop_sdiv`). +/// See [`signed_binop`] for the operands, the width and the mode gating. +/// +/// # Safety +/// `function_name` must be null or a valid null-terminated C string. +#[no_mangle] +pub unsafe extern "C" fn map2check_binop_div_signed( + a: i64, + b: i64, + bits: u32, + line: u32, + function_name: *const std::os::raw::c_char, +) { + ffi_guard( + // SAFETY: forwarded caller contract. + || unsafe { signed_binop(SignedOp::Div, a, b, bits, line, function_name) }, + (), + ); +} + +/// Signed `a % b`: reports `DIVBYZERO` when `b` is 0 and `OVERFLOW` for +/// MIN % -1 of `bits` (develop `map2check_binop_srem`). +/// See [`signed_binop`] for the operands, the width and the mode gating. +/// +/// # Safety +/// `function_name` must be null or a valid null-terminated C string. +#[no_mangle] +pub unsafe extern "C" fn map2check_binop_rem_signed( + a: i64, + b: i64, + bits: u32, + line: u32, + function_name: *const std::os::raw::c_char, +) { + ffi_guard( + // SAFETY: forwarded caller contract. + || unsafe { signed_binop(SignedOp::Rem, a, b, bits, line, function_name) }, + (), + ); +} + // ── Analysis mode: memory ───────────────────────────────────────────────────── /// Checks whether loading from `ptr` is valid. diff --git a/rust/src/ffi_golden_tests.rs b/rust/src/ffi_golden_tests.rs index 80aa46a..b47e302 100644 --- a/rust/src/ffi_golden_tests.rs +++ b/rust/src/ffi_golden_tests.rs @@ -301,6 +301,21 @@ fn overflow_matches_c_runtime() { assert_matches_golden("OVERFLOW", "overflow_child"); } +#[test] +fn div_by_zero_child() { + if is_child() { + start(AnalysisMode::Overflow); + // SAFETY: C-string literal is valid. + unsafe { map2check_binop_div_signed(7, 0, 32, 23, c"main".as_ptr()) }; + map2check_success(); + } +} + +#[test] +fn div_by_zero_matches_c_runtime() { + assert_matches_golden("DIVBYZERO", "div_by_zero_child"); +} + #[test] fn assert_child() { if is_child() { diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 02afdf4..580086d 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -702,6 +702,156 @@ fn binop_shr_unsigned_detects_wide_shift() { ); } +// ── Signed binops at the operation's width (Task 2.5, AnalysisModeOverflow.c) ── + +type SignedBinop = unsafe extern "C" fn(i64, i64, u32, u32, *const std::os::raw::c_char); + +/// What one call of a signed binop export records on a fresh analysis. +fn signed_outcome(op: SignedBinop, a: i64, b: i64, bits: u32) -> ViolatedProperty { + state::reset(); + // SAFETY: C-string literal is valid and null-terminated. + unsafe { op(a, b, bits, 4, c"f".as_ptr()) }; + let result = snapshot(|s| s.result.clone()); + if !result.ok { + assert_eq!((result.line, result.function_name.as_str()), (4, "f")); + } + result.property +} + +/// `signed_min_for_width` / `signed_max_for_width` of develop +/// (`AnalysisModeOverflow.c:111-124`). +fn signed_bounds(bits: u32) -> (i64, i64) { + if bits == 0 || bits >= 64 { + (i64::MIN, i64::MAX) + } else { + (-(1i64 << (bits - 1)), (1i64 << (bits - 1)) - 1) + } +} + +#[test] +fn signed_binops_check_every_width_like_develop() { + let _guard = fresh(); + let add: SignedBinop = map2check_binop_add_signed; + let sub: SignedBinop = map2check_binop_sub_signed; + let mul: SignedBinop = map2check_binop_mul_signed; + let div: SignedBinop = map2check_binop_div_signed; + let rem: SignedBinop = map2check_binop_rem_signed; + // 8/16/32/64 are what C types give; the pass sends any width up to 64 + // (OverflowPass.cpp:426), and develop treats 0 like 64. + for bits in [8, 16, 32, 64, 1, 24, 0] { + let (min, max) = signed_bounds(bits); + let cases: [(&str, SignedBinop, i64, i64, ViolatedProperty); 14] = [ + ("max+1", add, max, 1, P::Overflow), + ("(max-1)+1", add, max - 1, 1, P::None), + ("min-1", sub, min, 1, P::Overflow), + ("(min+1)-1", sub, min + 1, 1, P::None), + ("min*-1", mul, min, -1, P::Overflow), + ("max*-1", mul, max, -1, P::None), + ("min/-1", div, min, -1, P::Overflow), + ("max/-1", div, max, -1, P::None), + ("min%-1", rem, min, -1, P::Overflow), + ("max%-1", rem, max, -1, P::None), + ("x/0", div, max, 0, P::DivByZero), + ("x/1", div, max, 1, P::None), + ("x%0", rem, min, 0, P::DivByZero), + ("x%1", rem, min, 1, P::None), + ]; + for (name, op, a, b, expected) in cases { + assert_eq!( + signed_outcome(op, a, b, bits), + expected, + "{name} em {bits} bits ({a}, {b})" + ); + } + } +} + +#[test] +fn signed_binops_follow_develop_on_operands_wider_than_the_width() { + let _guard = fresh(); + let add: SignedBinop = map2check_binop_add_signed; + let mul: SignedBinop = map2check_binop_mul_signed; + let div: SignedBinop = map2check_binop_div_signed; + // 64-bit predicates first (AnalysisModeOverflow.c:141-183), then the + // range of the width: an i64 product that wraps is caught even at 8 bits. + assert_eq!(signed_outcome(mul, i64::MAX, 2, 8), P::Overflow); + assert_eq!(signed_outcome(add, i64::MIN, -1, 32), P::Overflow); + assert_eq!(signed_outcome(mul, -128, -1, 8), P::Overflow); + assert_eq!(signed_outcome(mul, 0, i64::MIN, 64), P::None); + // Division compares only the dividend with the width's minimum (:259), + // so MIN/-1 of a wider width is not an overflow at a narrower one. + assert_eq!(signed_outcome(div, i64::from(i32::MIN), -1, 16), P::None); + assert_eq!(signed_outcome(div, 0, 0, 32), P::DivByZero); +} + +#[test] +fn signed_binops_are_gated_like_the_other_binops() { + let _guard = fresh(); + let calls: [(SignedBinop, i64, i64); 3] = [ + (map2check_binop_add_signed, i64::from(i32::MAX), 1), + (map2check_binop_div_signed, 1, 0), + (map2check_binop_rem_signed, 1, 0), + ]; + for mode in MODES { + for (op, a, b) in calls { + state::reset(); + map2check_set_mode(mode as i32); + map2check_init(); + // SAFETY: C-string literal is valid and null-terminated. + unsafe { op(a, b, 32, 4, c"f".as_ptr()) }; + let reported = !snapshot(|s| s.result.ok); + assert_eq!(reported, mode == AnalysisMode::Overflow, "{mode:?}"); + } + } + state::reset_config(); + state::reset(); + // Unset mode: every check is active. + // SAFETY: C-string literal is valid and null-terminated. + unsafe { map2check_binop_div_signed(1, 0, 32, 4, c"f".as_ptr()) }; + assert_violated(ViolatedProperty::DivByZero, 4, "f"); +} + +#[test] +fn signed_binop_violation_is_final() { + let _guard = fresh(); + // SAFETY: C-string literals are valid and null-terminated. + unsafe { + map2check_binop_rem_signed(1, 0, 32, 4, c"f".as_ptr()); + map2check_binop_add_signed(i64::MAX, 1, 64, 9, c"g".as_ptr()); + } + assert_violated(ViolatedProperty::DivByZero, 4, "f"); +} + +#[test] +fn signed_binops_called_as_c_does() { + let _guard = fresh(); + // As the pass emits them: i32 operands sign-extended, width 32. + // SAFETY: C-string literals are valid and null-terminated. + unsafe { + map2check_binop_add_signed(-5, 3, 32, 1, c"main".as_ptr()); + map2check_binop_sub_signed(-5, 3, 32, 1, c"main".as_ptr()); + map2check_binop_mul_signed(-5, 3, 32, 1, c"main".as_ptr()); + map2check_binop_div_signed(-5, 3, 32, 1, c"main".as_ptr()); + map2check_binop_rem_signed(-5, 3, 32, 1, c"main".as_ptr()); + } + assert_ok(); + // SAFETY: as above. + unsafe { map2check_binop_sub_signed(i64::from(i16::MIN), 1, 16, 2, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::Overflow, 2, "main"); + state::reset(); + // SAFETY: as above. + unsafe { map2check_binop_mul_signed(i64::from(i8::MIN), -1, 8, 3, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::Overflow, 3, "main"); +} + +#[test] +fn set_false_result_accepts_div_by_zero() { + let _guard = fresh(); + // SAFETY: C-string literal is valid and null-terminated. + unsafe { set_false_result(10, 3, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::DivByZero, 3, "main"); +} + // ── Analysis mode: memory ─────────────────────────────────────────────────── #[test] diff --git a/rust/src/output/legacy_files.rs b/rust/src/output/legacy_files.rs index f8a81d2..4ef6c9c 100644 --- a/rust/src/output/legacy_files.rs +++ b/rust/src/output/legacy_files.rs @@ -114,6 +114,8 @@ fn property_text(result: &AnalysisResult) -> (Option, Option<&'static st Some("map2check_property_memcleanup"), ), ViolatedProperty::Reachability => (Some(located("TARGET-REACHED")), None), + // `:98-102`: no per-property file. + ViolatedProperty::DivByZero => (Some(located("FALSE-DIVBYZERO")), None), // The only UNKNOWN the C runtime reports is `write_property_unknown` // (NULL from malloc, AnalysisModeMemtrack.c:305-311): it writes // `map2check_property__unknown` and no `map2check_property`. @@ -665,6 +667,22 @@ mod tests { } } + #[test] + fn div_by_zero_writes_only_map2check_property() { + // PropertyGenerator.c:98-102: no per-property file for FALSE_DIVBYZERO. + let dir = tempfile::tempdir().unwrap(); + write_property_files( + dir.path(), + &violated(ViolatedProperty::DivByZero, 23, "main"), + ) + .unwrap(); + assert_eq!( + std::fs::read(dir.path().join("map2check_property")).unwrap(), + golden("DIVBYZERO", "map2check_property") + ); + assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 1); + } + #[test] fn unknown_uses_legacy_double_underscore_name() { // PropertyGenerator.c:16; SD-16. diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index 958d3e0..8c7e7a5 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -100,6 +100,11 @@ static void run_pre_init_boundary(void) { map2check_binop_shl_int(1, 1, 1, 0, "main"); map2check_binop_shr_int(1, 1, 1, 0, "main"); map2check_binop_shr_unsigned(1u, 1u, 1, 0, "main"); + map2check_binop_add_signed(1, 1, 32, 1, "main"); + map2check_binop_sub_signed(1, 1, 32, 1, "main"); + map2check_binop_mul_signed(1, 1, 32, 1, "main"); + map2check_binop_div_signed(1, 1, 32, 1, "main"); + map2check_binop_rem_signed(1, 1, 32, 1, "main"); (void)__VERIFIER_nondet_bool(); (void)__VERIFIER_nondet_char(); @@ -197,6 +202,13 @@ static void run_bad_inputs(void) { map2check_binop_shl_int(1, 31, 11, 0, NULL); map2check_binop_shr_int(-1, 32, 11, 0, NULL); map2check_binop_shr_unsigned(4294967295u, 32, 11, 0, NULL); + map2check_binop_add_signed(INT64_MAX, 1, 64, 12, NULL); + map2check_binop_sub_signed(INT64_MIN, 1, 64, 12, NULL); + map2check_binop_mul_signed(INT64_MIN, -1, 64, 12, NULL); + map2check_binop_div_signed(INT64_MIN, -1, 64, 12, NULL); + map2check_binop_rem_signed(INT64_MIN, -1, 64, 12, NULL); + map2check_binop_div_signed(1, 0, 0, 12, NULL); + map2check_binop_rem_signed(1, 0, 4096, 12, NULL); /* nondet generators, pre-init */ (void)__VERIFIER_nondet_bool(); @@ -270,6 +282,11 @@ static void run_valid_pass(void) { map2check_binop_shl_int(1, 2, 29, 0, "main"); map2check_binop_shr_int(8, 2, 29, 0, "main"); map2check_binop_shr_unsigned(8u, 2u, 29, 0, "main"); + map2check_binop_add_signed(1, 2, 32, 29, "main"); + map2check_binop_sub_signed(2, 1, 16, 29, "main"); + map2check_binop_mul_signed(2, 3, 8, 29, "main"); + map2check_binop_div_signed(6, 3, 64, 29, "main"); + map2check_binop_rem_signed(7, 3, 32, 29, "main"); (void)__VERIFIER_nondet_bool(); (void)__VERIFIER_nondet_char(); diff --git a/tests/legacy_golden/DIVBYZERO/.exit_status b/tests/legacy_golden/DIVBYZERO/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/DIVBYZERO/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/DIVBYZERO/klee_log.csv b/tests/legacy_golden/DIVBYZERO/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/DIVBYZERO/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/DIVBYZERO/map2check_checked_error b/tests/legacy_golden/DIVBYZERO/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/DIVBYZERO/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/DIVBYZERO/map2check_property b/tests/legacy_golden/DIVBYZERO/map2check_property new file mode 100644 index 0000000..3ca094b --- /dev/null +++ b/tests/legacy_golden/DIVBYZERO/map2check_property @@ -0,0 +1,3 @@ +FALSE-DIVBYZERO +Line: 23 +Function: main diff --git a/tests/legacy_golden/DIVBYZERO/track_bb_log.st b/tests/legacy_golden/DIVBYZERO/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/DIVBYZERO/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/README.md b/tests/legacy_golden/README.md index 050df4a..2669700 100644 --- a/tests/legacy_golden/README.md +++ b/tests/legacy_golden/README.md @@ -27,6 +27,7 @@ compiled with `clang-16`. Each binary runs in a fresh directory. | `NONE` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_free` of `p`, `map2check_success` | | `UNKNOWN` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_malloc(NULL, 4)` | | `OVERFLOW` | `AnalysisModeOverflow.c` | `map2check_binop_add(INT_MAX, 1, 32, ...)` | +| `DIVBYZERO` | `AnalysisModeOverflow.c` | `map2check_binop_sdiv(7, 0, 32, ...)` | | `ASSERT` | `AnalysisModeAssert.c` | `map2check_assert(0, ...)` | | `TARGET-REACHED` | `AnalysisModeNone.c` (`REACHABILITY_MODE`) | `map2check_target_function` | diff --git a/tests/legacy_golden/drivers/DIVBYZERO.c b/tests/legacy_golden/drivers/DIVBYZERO.c new file mode 100644 index 0000000..7284fce --- /dev/null +++ b/tests/legacy_golden/drivers/DIVBYZERO.c @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Overflow mode: 7 / 0 on a 32-bit int. + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + map2check_binop_sdiv(7, 0, 32, 23, 0, "main"); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/build_goldens.sh b/tests/legacy_golden/drivers/build_goldens.sh index fb84261..4b5f765 100755 --- a/tests/legacy_golden/drivers/build_goldens.sh +++ b/tests/legacy_golden/drivers/build_goldens.sh @@ -43,12 +43,13 @@ declare -A MODE=( [NONE]="AnalysisModeMemtrack $MEMORY" [UNKNOWN]="AnalysisModeMemtrack $MEMORY" [OVERFLOW]="AnalysisModeOverflow" + [DIVBYZERO]="AnalysisModeOverflow" [ASSERT]="AnalysisModeAssert" [TARGET-REACHED]="AnalysisModeNone" ) for tag in FALSE-FREE FALSE-DEREF FALSE-MEMTRACK FALSE-MEMCLEANUP NONE \ - UNKNOWN OVERFLOW ASSERT TARGET-REACHED; do + UNKNOWN OVERFLOW DIVBYZERO ASSERT TARGET-REACHED; do bc "$DRIVERS/$tag.c" "$WORK/$tag-output.bc" inputs=("$WORK/$tag-output.bc") for f in $COMMON ${MODE[$tag]} NonDetGeneratorLog WitnessGeneration; do diff --git a/tests/legacy_golden/drivers/golden.h b/tests/legacy_golden/drivers/golden.h index 4077b6a..5e1e3a9 100644 --- a/tests/legacy_golden/drivers/golden.h +++ b/tests/legacy_golden/drivers/golden.h @@ -53,6 +53,8 @@ void map2check_load(void *ptr, int size); void map2check_check_deref(int line, const char *function_name); void map2check_binop_add(long long param1, long long param2, unsigned width, unsigned line, unsigned scope, char *function_name); +void map2check_binop_sdiv(long long param1, long long param2, unsigned width, + unsigned line, unsigned scope, char *function_name); void map2check_assert(int condition, int line, const char *func_name); /* Fixed addresses, mirrored by rust/src/output/golden_tests.rs. */ From 6a2b915b493784c1a8442a35a4275177779cefb7 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 22:39:12 -0400 Subject: [PATCH 27/63] feat: track posix_memalign through the void ** it wrote (#13) Add map2check_map_posix_memalign(ptr_address, size), the core target of develop's map2check_posix (AnalysisModeMemtrack.c:327-333): reads the block from *ptr_address, records it as map2check_map_malloc and records the store into *ptr_address with the scope, line, name and function develop passes. Golden POSIX-MEMALIGN captured from develop. --- docs/api-deltas.md | 3 +- docs/cutover/abi-mapping.tsv | 2 +- rust/src/ffi.rs | 48 ++++++++++++++++ rust/src/ffi_golden_tests.rs | 50 ++++++++++++++++ rust/src/ffi_tests.rs | 57 +++++++++++++++++++ tests/ffi_harness.c | 14 +++++ .../legacy_golden/POSIX-MEMALIGN/.exit_status | 1 + .../POSIX-MEMALIGN/allocation_log.csv | 4 ++ .../legacy_golden/POSIX-MEMALIGN/heap_log.csv | 3 + .../legacy_golden/POSIX-MEMALIGN/klee_log.csv | 5 ++ .../legacy_golden/POSIX-MEMALIGN/list_log.csv | 2 + .../POSIX-MEMALIGN/map2check_checked_error | 1 + .../POSIX-MEMALIGN/map2check_property | 1 + .../map2check_property_memtrack | 1 + .../POSIX-MEMALIGN/track_bb_log.st | 2 + tests/legacy_golden/README.md | 1 + tests/legacy_golden/drivers/POSIX-MEMALIGN.c | 17 ++++++ tests/legacy_golden/drivers/build_goldens.sh | 3 +- tests/legacy_golden/drivers/golden.h | 3 + 19 files changed, 215 insertions(+), 3 deletions(-) create mode 100644 tests/legacy_golden/POSIX-MEMALIGN/.exit_status create mode 100644 tests/legacy_golden/POSIX-MEMALIGN/allocation_log.csv create mode 100644 tests/legacy_golden/POSIX-MEMALIGN/heap_log.csv create mode 100644 tests/legacy_golden/POSIX-MEMALIGN/klee_log.csv create mode 100644 tests/legacy_golden/POSIX-MEMALIGN/list_log.csv create mode 100644 tests/legacy_golden/POSIX-MEMALIGN/map2check_checked_error create mode 100644 tests/legacy_golden/POSIX-MEMALIGN/map2check_property create mode 100644 tests/legacy_golden/POSIX-MEMALIGN/map2check_property_memtrack create mode 100644 tests/legacy_golden/POSIX-MEMALIGN/track_bb_log.st create mode 100644 tests/legacy_golden/drivers/POSIX-MEMALIGN.c diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 1d28a27..fdc16aa 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -70,6 +70,7 @@ build with `--features libfuzzer`. | `map2check_map_funct_address` | `void map2check_map_funct_address(const char *var_name, const void *ptr_address)` | `container_memtracklog.h:48` | same | Signature and semantics reconstructed; see its section. | | `map2check_map_malloc` | `void map2check_map_malloc(const void *ptr_address, int32_t size)` | `map2check_map_malloc` | same | Negative sizes are ignored (SD-09). | | `map2check_map_calloc` | `void map2check_map_calloc(const void *ptr_address, int32_t quantity, int32_t size)` | `map2check_map_calloc` | same | Negative sizes are ignored (SD-09). | +| `map2check_map_posix_memalign` | `void map2check_map_posix_memalign(const void *ptr_address, int32_t size)` | `map2check_posix` (develop) | new | `ptr_address` is posix_memalign's `memptr` (`void **`): reads the block from it, records it as `map2check_map_malloc`, then the store of the block into `*ptr_address` with scope 1, line 1, name `pointer`, function `cocoiso`, as develop's `map2check_posix` (`AnalysisModeMemtrack.c:327-333`); golden `POSIX-MEMALIGN`. NULL `ptr_address` ignored (SD-09); NULL block: as `map2check_map_malloc`, and no store. | | `map2check_map_free` | `void map2check_map_free(const char *var_name, const void *ptr_address, uint32_t scope, uint32_t line_number, const char *function_name)` | `map2check_map_free` | same | | | `map2check_map_store_pointer` | `void map2check_map_store_pointer(const void *var_address, const void *value, uint32_t scope, const char *var_name, int32_t line_number, const char *funct_name)` | `map2check_map_store_pointer` | same | | | `map2check_is_valid_assert` | `void map2check_is_valid_assert(int32_t line_number, const char *function_name, int32_t expression)` | `map2check_is_valid_assert` | changed | A failed assertion reports `ASSERT` (before Task 2.3: `REACHABILITY`) and, with the mode unset, prints the JSON report; with a mode set it ends the run (“Result files”) and prints nothing. Active in unset and assert modes only (SD-14). | @@ -137,7 +138,7 @@ Map2Check. | `map2check_non_static_alloca` | `(ptr, ptr, i64, i64, i64, i64)` | pending | Core equivalent: `map2check_map_non_static_alloca` (#21). | | `map2check_malloc` | `(ptr, i64)` | pending | Core equivalent: `map2check_map_malloc` (#21). | | `map2check_calloc` | `(ptr, i64, i64)` | pending | Core equivalent: `map2check_map_calloc` (#21). | -| `map2check_posix` | `(ptr, i64)` | pending | Deprecated in the embedded C library (#21). | +| `map2check_posix` | `(ptr, i64)` | pending | Core equivalent: `map2check_map_posix_memalign(ptr, int32_t)`; the pass narrows the size (`coerceToParamType`) and switches to it in Task 3.2. | | `map2check_free` | `(ptr, ptr, i64, i64, ptr)` | pending | The pass passes the address of the pointer variable (#21). | | `map2check_free_resolved_address` | `(ptr, i64, ptr, i1)` | pending | Core equivalent: `map2check_check_free_resolved_address` (#21). | | `map2check_load` | `(ptr, i64)` | pending | Core `map2check_check_load` takes 5 arguments (#21). | diff --git a/docs/cutover/abi-mapping.tsv b/docs/cutover/abi-mapping.tsv index 7f89cef..567aaf4 100644 --- a/docs/cutover/abi-mapping.tsv +++ b/docs/cutover/abi-mapping.tsv @@ -25,7 +25,7 @@ map2check_load map2check_check_load - C: (ptr,i64 size) (MemoryTrackPass.cpp:770 map2check_check_deref - - check_load já registra a violação (ffi.rs:894, LoadChecker); C: (i64 line,ptr fn) (MemoryTrackPass.cpp:785) só lê a flag ERROR_DEREF e encerra (AnalysisModeMemtrack.c:297); o passe deixa de emitir; C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) map2check_free map2check_check_free+map2check_map_free - C: (ptr name,ptr var_slot,i64 scope,i64 line,ptr fn) (MemoryTrackPass.cpp:818) → núcleo (ambos): (name,ptr,scope:u32,line:u32,fn); semântica confirmada: o núcleo é chaveado no endereço de heap carregado (ffi_tests.rs:722-744), então o passe carrega *ptr antes da chamada e a ordem é check_free depois map_free; o passe não chama nenhum dos dois se o valor carregado for NULL, como o C (AnalysisModeMemtrack.c:342); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:352) map2check_free_resolved_address map2check_check_free_resolved_address - chamada única; registra a liberação quando a checagem passa (como o C, :178-182); check_free não registra — o passe chama map_free depois; vale também para a instrumentação de realloc (MemoryTrackPass.cpp:169-182); SD-06 (#37, corrigido): endereço sem registro reporta MEMSAFETY_FREE, como is_invalid_free (ListLog.c:95-118); SD-13: free de ponteiro nunca armazenado é válido no núcleo e FALSE_FREE no C; C: (ptr,i64 line,ptr fn,i1 nullValid) (MemoryTrackPass.cpp:774) → núcleo: (ptr,line:u32,fn,is_null_valid:i16); i1 passa a i16 (zext); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:170) -map2check_posix map2check_map_posix_memalign 2.5 chamado em MemoryTrackPass.cpp:120 para posix_memalign; não é obsoleto; C: (ptr,i64 size) (MemoryTrackPass.cpp:793); o C runtime desreferencia ptr (void**) e registra malloc+store_pointer (AnalysisModeMemtrack.c:328); núcleo proposto: (ptr_slot,size:i32) +map2check_posix map2check_map_posix_memalign 2.5 chamado em MemoryTrackPass.cpp:120 para posix_memalign; não é obsoleto; C: (ptr,i64 size) (MemoryTrackPass.cpp:793); o C runtime desreferencia ptr (void**) e registra malloc+store_pointer (AnalysisModeMemtrack.c:327-333); núcleo: (ptr_address,size:i32), lê o bloco de *ptr_address e grava malloc e store_pointer(ptr_address,bloco,1,"pointer",1,"cocoiso") como o C; o passe trunca o tamanho para i32 (coerceToParamType); golden POSIX-MEMALIGN map2check_wasm_malloc map2check_wasm_malloc 2.8 C: (i64,i64) (MemoryTrackPass.cpp:825) map2check_wasm_free map2check_wasm_free 2.8 C: (i64,i64) (MemoryTrackPass.cpp:829) map2check_wasm_check_access map2check_wasm_check_access 2.8 C: (i64,i64) (MemoryTrackPass.cpp:833) diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index d570733..de89369 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -831,6 +831,54 @@ pub unsafe extern "C" fn map2check_map_calloc( ); } +/// Records a `posix_memalign(memptr, alignment, size)` that the program just +/// made: `ptr_address` is `memptr`, the `void **` the block was written +/// through. As develop's `map2check_posix` (`AnalysisModeMemtrack.c:327-333`), +/// reads the block from `*ptr_address`, records it as `map2check_map_malloc` +/// does, then records the store of the block into `*ptr_address` with +/// scope 1, line 1, variable name `pointer` and function `cocoiso`, the +/// values develop passes. A NULL `ptr_address` is ignored (SD-09); a NULL +/// block reports `UNKNOWN` in the memtrack and memcleanup modes and records +/// no store, as `map2check_map_malloc` does. +/// +/// # Safety +/// `ptr_address` must be null or point to a readable `void *`. +#[no_mangle] +pub unsafe extern "C" fn map2check_map_posix_memalign( + ptr_address: *const std::os::raw::c_void, + size: i32, +) { + ffi_guard( + || { + if ptr_address.is_null() { + return; + } + let slot = ptr_address.cast::<*const std::os::raw::c_void>(); + // SAFETY: caller guarantees a non-null `ptr_address` points to a + // readable `void *` (posix_memalign's `memptr`); the read is + // unaligned-safe. + let block = unsafe { slot.read_unaligned() }; + if malloc_returned_null(block) { + return; + } + // SAFETY: `block` is only stored; the names are NUL-terminated + // byte literals (cbindgen cannot parse `c""` literals). + unsafe { + map2check_map_malloc(block, size); + map2check_map_store_pointer( + ptr_address, + block, + 1, + b"pointer\0".as_ptr().cast(), + 1, + b"cocoiso\0".as_ptr().cast(), + ); + } + }, + (), + ); +} + /// # Safety /// `var_name` and `function_name` must be null or valid null-terminated C strings. #[no_mangle] diff --git a/rust/src/ffi_golden_tests.rs b/rust/src/ffi_golden_tests.rs index b47e302..bd6a61f 100644 --- a/rust/src/ffi_golden_tests.rs +++ b/rust/src/ffi_golden_tests.rs @@ -31,6 +31,35 @@ const SLOT_P: usize = ARENA; const FN_MAIN: usize = ARENA + 0x40; const BLOCK: usize = ARENA + 0x100; const WILD: usize = ARENA + 0x800; +const SLOT_Q: usize = ARENA + 0x8; +const BLOCK2: usize = ARENA + 0x200; +const ARENA_SIZE: usize = 4096; + +extern "C" { + fn mmap(addr: *mut c_void, len: usize, prot: i32, flags: i32, fd: i32, off: i64) + -> *mut c_void; +} + +/// `golden_arena()`: maps the arena at its fixed address, for the scenarios +/// whose export reads memory (`map2check_map_posix_memalign` reads the +/// `void *` slot, as develop's `map2check_posix` does). +fn map_arena() { + const PROT_READ_WRITE: i32 = 0x1 | 0x2; + const MAP_PRIVATE_ANONYMOUS_FIXED_NOREPLACE: i32 = 0x02 | 0x20 | 0x10_0000; + // SAFETY: an anonymous private mapping at an address that must be free + // (MAP_FIXED_NOREPLACE fails instead of replacing a mapping). + let arena = unsafe { + mmap( + ARENA as *mut c_void, + ARENA_SIZE, + PROT_READ_WRITE, + MAP_PRIVATE_ANONYMOUS_FIXED_NOREPLACE, + -1, + 0, + ) + }; + assert_eq!(arena as usize, ARENA, "não foi possível mapear a arena"); +} fn addr(a: usize) -> *const c_void { a as *const c_void @@ -316,6 +345,27 @@ fn div_by_zero_matches_c_runtime() { assert_matches_golden("DIVBYZERO", "div_by_zero_child"); } +#[test] +fn posix_memalign_child() { + if is_child() { + map_arena(); + start(AnalysisMode::Memtrack); + memory(); + // SAFETY: SLOT_Q lies in the mapped arena; the names are C strings. + unsafe { + map2check_map_alloca(c"q".as_ptr(), addr(SLOT_Q), 8, 8, 15, 1); + (SLOT_Q as *mut usize).write(BLOCK2); + map2check_map_posix_memalign(addr(SLOT_Q), 32); + } + map2check_success(); + } +} + +#[test] +fn posix_memalign_matches_c_runtime() { + assert_matches_golden("POSIX-MEMALIGN", "posix_memalign_child"); +} + #[test] fn assert_child() { if is_child() { diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 580086d..2ea821f 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -376,6 +376,63 @@ fn map_alloca_records_static_stack_slot() { assert!(!e.is_dynamic && !e.is_free); } +#[test] +fn posix_memalign_tracks_the_block_and_the_store_like_develop() { + // develop `map2check_posix` (AnalysisModeMemtrack.c:327-333): reads the + // block through the `void **` the program passed to posix_memalign, then + // `map2check_malloc(block, size)` and + // `map2check_add_store_pointer(slot, block, 1, "pointer", 1, "cocoiso")`. + let _guard = fresh(); + let mut slot: *const c_void = addr(0x4000); + let slot_address = std::ptr::addr_of_mut!(slot) as usize; + // SAFETY: `slot` is a live `void *` for the whole call. + unsafe { map2check_map_posix_memalign(addr(slot_address), 64) }; + let (malloc, store) = snapshot(|s| { + assert_eq!(s.memtrack.len(), 2); + (s.memtrack[0].clone(), s.memtrack[1].clone()) + }); + assert_eq!(malloc.origin, crate::memtrack::MemOrigin::Malloc); + assert_eq!((malloc.points_to, malloc.size_destiny), (0x4000, 64)); + assert!(malloc.is_dynamic && !malloc.is_free); + assert_eq!(store.origin, crate::memtrack::MemOrigin::StorePointer); + assert_eq!((store.var_addr, store.points_to), (slot_address, 0x4000)); + assert_eq!((store.scope, store.line), (1, 1)); + assert_eq!( + (store.ptr_name.as_str(), store.function_name.as_str()), + ("pointer", "cocoiso") + ); + // The block is live: a load inside it is valid, one past it is not. + // SAFETY: C-string literal is valid; fake addresses are never dereferenced. + unsafe { map2check_check_load(addr(0x4000 + 60), 2, 0, 4, c"main".as_ptr()) }; + assert_ok(); + // SAFETY: as above. + unsafe { map2check_check_load(addr(0x4000 + 64), 3, 0, 4, c"main".as_ptr()) }; + assert_violated(ViolatedProperty::MemsafetyDeref, 3, "main"); +} + +#[test] +fn posix_memalign_with_a_null_slot_is_ignored() { + let _guard = fresh(); + // SAFETY: a NULL slot is never dereferenced (SD-09). + unsafe { map2check_map_posix_memalign(std::ptr::null(), 64) }; + assert!(snapshot(|s| s.memtrack.is_empty())); + assert_ok(); +} + +#[test] +fn posix_memalign_null_block_is_unknown_in_memory_modes() { + // As `map2check_malloc(NULL, size)` (AnalysisModeMemtrack.c:305-311): + // UNKNOWN, and no store follows. + let _guard = fresh(); + map2check_set_mode(AnalysisMode::Memtrack as i32); + map2check_init(); + let mut slot: *const c_void = std::ptr::null(); + // SAFETY: `slot` is a live `void *` for the whole call. + unsafe { map2check_map_posix_memalign(std::ptr::addr_of_mut!(slot).cast(), 64) }; + assert_violated(ViolatedProperty::Unknown, 0, ""); + assert!(snapshot(|s| s.memtrack.is_empty())); +} + #[test] fn map_alloca_ignores_negative_size() { let _guard = fresh(); diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index 8c7e7a5..77ad14e 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -173,6 +173,11 @@ static void run_bad_inputs(void) { map2check_map_funct_address(NULL, NULL); map2check_map_malloc(NULL, -1); map2check_map_calloc(NULL, -1, -1); + map2check_map_posix_memalign(NULL, 16); + { + void *null_block = NULL; + map2check_map_posix_memalign(&null_block, -1); + } map2check_map_store_pointer(NULL, NULL, 0, NULL, 6, NULL); map2check_map_free(NULL, NULL, 0, 7, NULL); @@ -251,6 +256,15 @@ static void run_valid_pass(void) { map2check_map_non_static_alloca("stack", NULL, 8, 8, 20, 1); map2check_map_funct_address("main", (void *)0x1); map2check_map_store_pointer((void *)0x1, (void *)0x2, 1, "buf", 21, "main"); + { + /* posix_memalign(&aligned, 16, 32): the export reads the block + * through the void ** it receives. */ + void *aligned = NULL; + if (posix_memalign(&aligned, 16, 32) == 0) { + map2check_map_posix_memalign(&aligned, 32); + free(aligned); + } + } map2check_map_free("buf", (void *)0x1, 1, 22, "main"); map2check_check_load((void *)0x1, 23, 1, 4, "main"); diff --git a/tests/legacy_golden/POSIX-MEMALIGN/.exit_status b/tests/legacy_golden/POSIX-MEMALIGN/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/POSIX-MEMALIGN/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/POSIX-MEMALIGN/allocation_log.csv b/tests/legacy_golden/POSIX-MEMALIGN/allocation_log.csv new file mode 100644 index 0000000..a6555d1 --- /dev/null +++ b/tests/legacy_golden/POSIX-MEMALIGN/allocation_log.csv @@ -0,0 +1,4 @@ +0x10000100;16;0 +0x10000200;32;0 +0x10000100;0;1 +0x10000200;0;1 diff --git a/tests/legacy_golden/POSIX-MEMALIGN/heap_log.csv b/tests/legacy_golden/POSIX-MEMALIGN/heap_log.csv new file mode 100644 index 0000000..5500e2f --- /dev/null +++ b/tests/legacy_golden/POSIX-MEMALIGN/heap_log.csv @@ -0,0 +1,3 @@ +0x10000000;p;8 +0x10000040;main;1 +0x10000008;q;8 diff --git a/tests/legacy_golden/POSIX-MEMALIGN/klee_log.csv b/tests/legacy_golden/POSIX-MEMALIGN/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/POSIX-MEMALIGN/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/POSIX-MEMALIGN/list_log.csv b/tests/legacy_golden/POSIX-MEMALIGN/list_log.csv new file mode 100644 index 0000000..55fba1c --- /dev/null +++ b/tests/legacy_golden/POSIX-MEMALIGN/list_log.csv @@ -0,0 +1,2 @@ +0;0x10000000;0x10000100;1;0;1;p;11;main;5 +0;0x10000008;0x10000200;1;0;1;pointer;1;cocoiso;6 diff --git a/tests/legacy_golden/POSIX-MEMALIGN/map2check_checked_error b/tests/legacy_golden/POSIX-MEMALIGN/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/POSIX-MEMALIGN/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/POSIX-MEMALIGN/map2check_property b/tests/legacy_golden/POSIX-MEMALIGN/map2check_property new file mode 100644 index 0000000..32e1ea0 --- /dev/null +++ b/tests/legacy_golden/POSIX-MEMALIGN/map2check_property @@ -0,0 +1 @@ +FALSE-MEMTRACK diff --git a/tests/legacy_golden/POSIX-MEMALIGN/map2check_property_memtrack b/tests/legacy_golden/POSIX-MEMALIGN/map2check_property_memtrack new file mode 100644 index 0000000..32e1ea0 --- /dev/null +++ b/tests/legacy_golden/POSIX-MEMALIGN/map2check_property_memtrack @@ -0,0 +1 @@ +FALSE-MEMTRACK diff --git a/tests/legacy_golden/POSIX-MEMALIGN/track_bb_log.st b/tests/legacy_golden/POSIX-MEMALIGN/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/POSIX-MEMALIGN/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/README.md b/tests/legacy_golden/README.md index 2669700..386afdb 100644 --- a/tests/legacy_golden/README.md +++ b/tests/legacy_golden/README.md @@ -28,6 +28,7 @@ compiled with `clang-16`. Each binary runs in a fresh directory. | `UNKNOWN` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_malloc(NULL, 4)` | | `OVERFLOW` | `AnalysisModeOverflow.c` | `map2check_binop_add(INT_MAX, 1, 32, ...)` | | `DIVBYZERO` | `AnalysisModeOverflow.c` | `map2check_binop_sdiv(7, 0, 32, ...)` | +| `POSIX-MEMALIGN` | `AnalysisModeMemtrack.c` | memory prelude, `q` on the stack, `q = BLOCK2` written into its slot, `map2check_posix(&q, 32)`, `map2check_success` (both blocks leak) | | `ASSERT` | `AnalysisModeAssert.c` | `map2check_assert(0, ...)` | | `TARGET-REACHED` | `AnalysisModeNone.c` (`REACHABILITY_MODE`) | `map2check_target_function` | diff --git a/tests/legacy_golden/drivers/POSIX-MEMALIGN.c b/tests/legacy_golden/drivers/POSIX-MEMALIGN.c new file mode 100644 index 0000000..d715de3 --- /dev/null +++ b/tests/legacy_golden/drivers/POSIX-MEMALIGN.c @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Memtrack mode: posix_memalign(&q, 16, 32) next to p = malloc(16); both + * blocks leak. + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + golden_memory(); + map2check_alloca("q", SLOT_Q, 8, 8, 15, 1); + *(void **)SLOT_Q = BLOCK2; + map2check_posix(SLOT_Q, 32); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/build_goldens.sh b/tests/legacy_golden/drivers/build_goldens.sh index 4b5f765..b1256de 100755 --- a/tests/legacy_golden/drivers/build_goldens.sh +++ b/tests/legacy_golden/drivers/build_goldens.sh @@ -44,12 +44,13 @@ declare -A MODE=( [UNKNOWN]="AnalysisModeMemtrack $MEMORY" [OVERFLOW]="AnalysisModeOverflow" [DIVBYZERO]="AnalysisModeOverflow" + [POSIX-MEMALIGN]="AnalysisModeMemtrack $MEMORY" [ASSERT]="AnalysisModeAssert" [TARGET-REACHED]="AnalysisModeNone" ) for tag in FALSE-FREE FALSE-DEREF FALSE-MEMTRACK FALSE-MEMCLEANUP NONE \ - UNKNOWN OVERFLOW DIVBYZERO ASSERT TARGET-REACHED; do + UNKNOWN OVERFLOW DIVBYZERO POSIX-MEMALIGN ASSERT TARGET-REACHED; do bc "$DRIVERS/$tag.c" "$WORK/$tag-output.bc" inputs=("$WORK/$tag-output.bc") for f in $COMMON ${MODE[$tag]} NonDetGeneratorLog WitnessGeneration; do diff --git a/tests/legacy_golden/drivers/golden.h b/tests/legacy_golden/drivers/golden.h index 5e1e3a9..42612db 100644 --- a/tests/legacy_golden/drivers/golden.h +++ b/tests/legacy_golden/drivers/golden.h @@ -41,6 +41,7 @@ void map2check_alloca(const char *name, void *ptr, int size, int size_of_primitive, int line, int scope); void map2check_function(const char *name, void *ptr); void map2check_malloc(void *ptr, int size); +void map2check_posix(void *ptr, int size); void map2check_add_store_pointer(void *var, void *value, unsigned scope, const char *name, int line, const char *function_name); @@ -64,6 +65,8 @@ void map2check_assert(int condition, int line, const char *func_name); #define FN_MAIN ((void *)(GOLDEN_ARENA + 0x40)) /* address of main */ #define BLOCK ((void *)(GOLDEN_ARENA + 0x100)) /* heap block, 16 bytes */ #define WILD ((void *)(GOLDEN_ARENA + 0x800)) /* never allocated */ +#define SLOT_Q ((void *)(GOLDEN_ARENA + 0x8)) /* the pointer variable q */ +#define BLOCK2 ((void *)(GOLDEN_ARENA + 0x200)) /* heap block, 32 bytes */ void __real_free(void *ptr); void __wrap_free(void *ptr) { From 52b9a259b107559f38b8f3b38e1ca237a4c6ff3d Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 22:45:00 -0400 Subject: [PATCH 28/63] feat: pass the pointer variable to the free exports so list_log.csv names it (#13, #19) map2check_check_free and map2check_map_free take (name, var_address, ptr, scope, line, fn): var_address is the slot develop's map2check_free receives (MemoryTrackPass.cpp:818), ptr stays the block the checks are keyed on. The FREE row of list_log.csv and the memcleanup verdict now use the real variable, and a failing check keeps the row develop appends before it checks (AnalysisModeMemtrack.c:346-358). Golden FALSE-FREE now covers the failing map2check_free path; the old scenario is kept as FALSE-FREE-RESOLVED. SD-17 loses its two free cases. --- TODO.md | 11 +- docs/api-deltas.md | 6 +- docs/cutover/abi-mapping.tsv | 2 +- docs/integration-guide.md | 3 +- docs/semantic-deltas.md | 30 +- rust/src/ffi.rs | 70 +++- rust/src/ffi_golden_tests.rs | 49 ++- rust/src/ffi_tests.rs | 384 ++++++++++++++++-- rust/src/memtrack/mod.rs | 44 +- rust/src/output/legacy_files.rs | 43 +- rust/src/output/termination.rs | 7 +- rust/src/state.rs | 5 + test/codeAPI_by_caller_test.c | 7 +- tests/compat_docs.rs | 9 +- tests/ffi_harness.c | 14 +- .../FALSE-FREE-RESOLVED/.exit_status | 1 + .../FALSE-FREE-RESOLVED/allocation_log.csv | 1 + .../FALSE-FREE-RESOLVED/heap_log.csv | 2 + .../FALSE-FREE-RESOLVED/klee_log.csv | 5 + .../FALSE-FREE-RESOLVED/list_log.csv | 2 + .../map2check_checked_error | 1 + .../FALSE-FREE-RESOLVED/map2check_property | 3 + .../map2check_property_free | 3 + .../FALSE-FREE-RESOLVED/track_bb_log.st | 2 + tests/legacy_golden/FALSE-FREE/list_log.csv | 1 + tests/legacy_golden/README.md | 3 +- .../drivers/FALSE-FREE-RESOLVED.c | 16 + tests/legacy_golden/drivers/FALSE-FREE.c | 6 +- tests/legacy_golden/drivers/build_goldens.sh | 3 +- 29 files changed, 621 insertions(+), 112 deletions(-) create mode 100644 tests/legacy_golden/FALSE-FREE-RESOLVED/.exit_status create mode 100644 tests/legacy_golden/FALSE-FREE-RESOLVED/allocation_log.csv create mode 100644 tests/legacy_golden/FALSE-FREE-RESOLVED/heap_log.csv create mode 100644 tests/legacy_golden/FALSE-FREE-RESOLVED/klee_log.csv create mode 100644 tests/legacy_golden/FALSE-FREE-RESOLVED/list_log.csv create mode 100644 tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_checked_error create mode 100644 tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_property create mode 100644 tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_property_free create mode 100644 tests/legacy_golden/FALSE-FREE-RESOLVED/track_bb_log.st create mode 100644 tests/legacy_golden/drivers/FALSE-FREE-RESOLVED.c diff --git a/TODO.md b/TODO.md index 97a2cd1..ad3fccc 100644 --- a/TODO.md +++ b/TODO.md @@ -163,11 +163,12 @@ e não repetem as tarefas já listadas. núcleo; incluir a matriz completa de símbolos e resolver/documentar a colisão de assinatura de `map2check_check_deref`. Excluir explicitamente símbolos wasm se permanecerem fora do escopo. - Ao passar o endereço da variável-ponteiro em `map2check_free`, fechar a - parte do SD-17 sobre `list_log.csv`: a linha FREE com a variável certa e a - linha que o develop acrescenta antes de um `free` inválido, com um golden - do caminho `map2check_free` que falha (hoje o golden FALSE-FREE usa - `free_resolved_address`; ver `tests/legacy_golden/README.md`). + A parte do SD-17 sobre `list_log.csv` foi fechada na Task 2.5: + `map2check_check_free`/`map2check_map_free` recebem o endereço da + variável-ponteiro, a linha FREE nomeia a variável certa e um `free` + inválido mantém a linha que o develop acrescenta antes de checar (golden + `FALSE-FREE` pelo caminho `map2check_free`; o antigo virou + `FALSE-FREE-RESOLVED`). - [x] **Compatibilizar o canal de resultado legado.** Feito na Task 2.4 (#19): `rust/src/output/legacy_files.rs` e golden tests em diff --git a/docs/api-deltas.md b/docs/api-deltas.md index fdc16aa..9cbedb6 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -71,7 +71,7 @@ build with `--features libfuzzer`. | `map2check_map_malloc` | `void map2check_map_malloc(const void *ptr_address, int32_t size)` | `map2check_map_malloc` | same | Negative sizes are ignored (SD-09). | | `map2check_map_calloc` | `void map2check_map_calloc(const void *ptr_address, int32_t quantity, int32_t size)` | `map2check_map_calloc` | same | Negative sizes are ignored (SD-09). | | `map2check_map_posix_memalign` | `void map2check_map_posix_memalign(const void *ptr_address, int32_t size)` | `map2check_posix` (develop) | new | `ptr_address` is posix_memalign's `memptr` (`void **`): reads the block from it, records it as `map2check_map_malloc`, then the store of the block into `*ptr_address` with scope 1, line 1, name `pointer`, function `cocoiso`, as develop's `map2check_posix` (`AnalysisModeMemtrack.c:327-333`); golden `POSIX-MEMALIGN`. NULL `ptr_address` ignored (SD-09); NULL block: as `map2check_map_malloc`, and no store. | -| `map2check_map_free` | `void map2check_map_free(const char *var_name, const void *ptr_address, uint32_t scope, uint32_t line_number, const char *function_name)` | `map2check_map_free` | same | | +| `map2check_map_free` | `void map2check_map_free(const char *var_name, const void *var_address, const void *ptr_address, uint32_t scope, uint32_t line_number, const char *function_name)` | `map2check_map_free` | changed | Task 2.5: gains `var_address`, the address of the pointer variable freed (the slot develop's `map2check_free` receives); `ptr_address` stays the block, which the release is keyed on. The slot only names the variable in the FREE row of `list_log.csv` (`AnalysisModeMemtrack.c:339-347`). | | `map2check_map_store_pointer` | `void map2check_map_store_pointer(const void *var_address, const void *value, uint32_t scope, const char *var_name, int32_t line_number, const char *funct_name)` | `map2check_map_store_pointer` | same | | | `map2check_is_valid_assert` | `void map2check_is_valid_assert(int32_t line_number, const char *function_name, int32_t expression)` | `map2check_is_valid_assert` | changed | A failed assertion reports `ASSERT` (before Task 2.3: `REACHABILITY`) and, with the mode unset, prints the JSON report; with a mode set it ends the run (“Result files”) and prints nothing. Active in unset and assert modes only (SD-14). | | `map2check_binop_add_int` | `void map2check_binop_add_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name)` | `map2check_binop_add_int` | same | `int`/`unsigned` (32-bit) only, as in C. Active in unset and overflow modes only (SD-14). | @@ -92,7 +92,7 @@ build with `--features libfuzzer`. | `map2check_binop_div_signed` | `void map2check_binop_div_signed(int64_t a, int64_t b, uint32_t bits, uint32_t line, const char *function_name)` | `map2check_binop_sdiv` (develop) | changed | Operands sign-extended to 64 bits, `bits` the width of the result type (0 or above 64 means 64), as the pass sends them; reports `DIVBYZERO` (`ViolatedProperty` 10, develop `FALSE-DIVBYZERO`) when `b` is 0, `OVERFLOW` for MIN / -1 of `bits` (`AnalysisModeOverflow.c:110-276`). No `scope` argument. Records and continues with the mode unset; ends the run with a mode set (SD-03). Active in unset and overflow modes only (SD-14). | | `map2check_binop_rem_signed` | `void map2check_binop_rem_signed(int64_t a, int64_t b, uint32_t bits, uint32_t line, const char *function_name)` | `map2check_binop_srem` (develop) | changed | Operands sign-extended to 64 bits, `bits` the width of the result type (0 or above 64 means 64), as the pass sends them; reports `DIVBYZERO` when `b` is 0, `OVERFLOW` for MIN % -1 of `bits` (`AnalysisModeOverflow.c:110-276`). No `scope` argument. Records and continues with the mode unset; ends the run with a mode set (SD-03). Active in unset and overflow modes only (SD-14). | | `map2check_check_load` | `void map2check_check_load(const void *ptr, int32_t line, uint32_t scope, int32_t size, const char *function_name)` | `map2check_check_load` | changed | Address-validity rule rewritten (plan R1; differential check in #15); since #11 a heap block is valid only while its latest alloc/free event is an allocation, as in `develop` (`AllocationLog.c:127-148`). The first violation is final; a later check never clears or replaces it (SD-03); NULL handling in SD-05. Active in unset and memtrack modes only (SD-14). | -| `map2check_check_free` | `void map2check_check_free(const char *_name, const void *ptr, uint32_t _scope, uint32_t line, const char *function_name)` | `map2check_check_free` | same | `_name` and `_scope` are ignored. Untracked address is a violation (SD-06). Active in unset and memtrack modes only (SD-14). | +| `map2check_check_free` | `void map2check_check_free(const char *name, const void *var_address, const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_free` | changed | Task 2.5: same arguments as `map2check_map_free` (gains `var_address`; `name` and `scope` are now used): on a first violation it keeps the FREE row develop's `map2check_free` appends to `list_log.csv` before it fails (`AnalysisModeMemtrack.c:346-358`). The check itself is unchanged and keyed on `ptr`. Untracked address is a violation (SD-06). Active in unset and memtrack modes only (SD-14). | | `map2check_check_free_resolved_address` | `void map2check_check_free_resolved_address(const void *ptr, uint32_t line, const char *function_name, int16_t is_null_valid)` | `analysis_memory.h:46` | same | Semantics in its section; records the release when the check passes, unlike `map2check_check_free`; untracked address is a violation (SD-06); see also SD-13. The check is active in unset and memtrack modes only; when inactive the release is still recorded (SD-14). | | `map2check_check_deref` | `void map2check_check_deref(const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_deref` | same | Name collides with the pass ABI `map2check_check_deref(i64, ptr)`; see the pass matrix (#21). Active in unset and memtrack modes only (SD-14). | | `map2check_check_mem_endprog` | `void map2check_check_mem_endprog(void)` | `map2check_check_mem_endprog` | changed | Leak algorithm rewritten. Runs `analysis_is_program_correct` of the mode: unset and memtrack report memtrack or memcleanup by option, memcleanup mode reports memcleanup only for a still-referenced last leak, the other modes nothing (SD-04, SD-14). | @@ -139,7 +139,7 @@ Map2Check. | `map2check_malloc` | `(ptr, i64)` | pending | Core equivalent: `map2check_map_malloc` (#21). | | `map2check_calloc` | `(ptr, i64, i64)` | pending | Core equivalent: `map2check_map_calloc` (#21). | | `map2check_posix` | `(ptr, i64)` | pending | Core equivalent: `map2check_map_posix_memalign(ptr, int32_t)`; the pass narrows the size (`coerceToParamType`) and switches to it in Task 3.2. | -| `map2check_free` | `(ptr, ptr, i64, i64, ptr)` | pending | The pass passes the address of the pointer variable (#21). | +| `map2check_free` | `(ptr, ptr, i64, i64, ptr)` | pending | Core equivalent: `map2check_check_free` then `map2check_map_free`, both `(name, var_slot, *var_slot, scope, line, fn)`; the pass loads the block and skips both calls for NULL (Task 3.2). | | `map2check_free_resolved_address` | `(ptr, i64, ptr, i1)` | pending | Core equivalent: `map2check_check_free_resolved_address` (#21). | | `map2check_load` | `(ptr, i64)` | pending | Core `map2check_check_load` takes 5 arguments (#21). | | `map2check_add_store_pointer` | 6 arguments | pending | Core `map2check_map_store_pointer` differs in order and types (#21). | diff --git a/docs/cutover/abi-mapping.tsv b/docs/cutover/abi-mapping.tsv index 567aaf4..3650649 100644 --- a/docs/cutover/abi-mapping.tsv +++ b/docs/cutover/abi-mapping.tsv @@ -23,7 +23,7 @@ map2check_function map2check_map_funct_address - C: (ptr name,ptr addr) (MemoryT map2check_add_store_pointer map2check_map_store_pointer - C: (ptr var,ptr value,i64 scope,ptr name,i64 line,ptr fn) (MemoryTrackPass.cpp:779) → núcleo: (var_address,value,scope:u32,var_name,line:i32,fn); mesma ordem map2check_load map2check_check_load - C: (ptr,i64 size) (MemoryTrackPass.cpp:770) → núcleo: (ptr,line:i32,scope:u32,size:i32,fn); o passe passa a emitir linha, escopo e função (DebugInfo); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (a flag ERROR_DEREF de AnalysisModeMemtrack.c:158 só encerra em check_deref); check_load grava a violação direto map2check_check_deref - - check_load já registra a violação (ffi.rs:894, LoadChecker); C: (i64 line,ptr fn) (MemoryTrackPass.cpp:785) só lê a flag ERROR_DEREF e encerra (AnalysisModeMemtrack.c:297); o passe deixa de emitir; C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) -map2check_free map2check_check_free+map2check_map_free - C: (ptr name,ptr var_slot,i64 scope,i64 line,ptr fn) (MemoryTrackPass.cpp:818) → núcleo (ambos): (name,ptr,scope:u32,line:u32,fn); semântica confirmada: o núcleo é chaveado no endereço de heap carregado (ffi_tests.rs:722-744), então o passe carrega *ptr antes da chamada e a ordem é check_free depois map_free; o passe não chama nenhum dos dois se o valor carregado for NULL, como o C (AnalysisModeMemtrack.c:342); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:352) +map2check_free map2check_check_free+map2check_map_free - C: (ptr name,ptr var_slot,i64 scope,i64 line,ptr fn) (MemoryTrackPass.cpp:818) → núcleo (ambos, Task 2.5): (name,var_slot,ptr,scope:u32,line:u32,fn), var_slot no lugar em que o passe já o envia e ptr = *var_slot; o núcleo é chaveado no endereço de heap carregado, então o passe carrega *var_slot antes da chamada e a ordem é check_free depois map_free; var_slot só nomeia a variável na linha FREE de list_log.csv (AnalysisModeMemtrack.c:339-347), e check_free guarda essa linha quando falha, como o C, que a acrescenta antes de checar (:346-358); golden FALSE-FREE; o passe não chama nenhum dos dois se o valor carregado for NULL, como o C (AnalysisModeMemtrack.c:342); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:352) map2check_free_resolved_address map2check_check_free_resolved_address - chamada única; registra a liberação quando a checagem passa (como o C, :178-182); check_free não registra — o passe chama map_free depois; vale também para a instrumentação de realloc (MemoryTrackPass.cpp:169-182); SD-06 (#37, corrigido): endereço sem registro reporta MEMSAFETY_FREE, como is_invalid_free (ListLog.c:95-118); SD-13: free de ponteiro nunca armazenado é válido no núcleo e FALSE_FREE no C; C: (ptr,i64 line,ptr fn,i1 nullValid) (MemoryTrackPass.cpp:774) → núcleo: (ptr,line:u32,fn,is_null_valid:i16); i1 passa a i16 (zext); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:170) map2check_posix map2check_map_posix_memalign 2.5 chamado em MemoryTrackPass.cpp:120 para posix_memalign; não é obsoleto; C: (ptr,i64 size) (MemoryTrackPass.cpp:793); o C runtime desreferencia ptr (void**) e registra malloc+store_pointer (AnalysisModeMemtrack.c:327-333); núcleo: (ptr_address,size:i32), lê o bloco de *ptr_address e grava malloc e store_pointer(ptr_address,bloco,1,"pointer",1,"cocoiso") como o C; o passe trunca o tamanho para i32 (coerceToParamType); golden POSIX-MEMALIGN map2check_wasm_malloc map2check_wasm_malloc 2.8 C: (i64,i64) (MemoryTrackPass.cpp:825) diff --git a/docs/integration-guide.md b/docs/integration-guide.md index 0068676..ddfd71e 100644 --- a/docs/integration-guide.md +++ b/docs/integration-guide.md @@ -148,7 +148,8 @@ print: `Container_AllocationLog` appears only for memory-safety violations (SD-02). `map2check_map_malloc((void *)0x1000, 16)` followed by -`map2check_check_free("p", (void *)0x2000, 0, 20, "main")` prints: +`map2check_check_free("p", (void *)0x10, (void *)0x2000, 0, 20, "main")` +(free of `p`, at `0x10`, holding `0x2000`) prints: ```json diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 759c5d7..981aad9 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -517,23 +517,19 @@ map2check_map_malloc(NULL, 4); /* writes map2check_property__unknown, aborts * ### SD-17 — The memory and nondet logs are rebuilt from this library's records **Classification:** accepted limitation -**Tests:** `rust/src/ffi_golden_tests.rs::false_free_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::none_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::false_memtrack_matches_c_runtime`, `rust/src/output/legacy_files.rs::nondet_rows_use_develop_types_and_value_formats` +**Tests:** `rust/src/ffi_golden_tests.rs::false_free_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::false_free_resolved_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::none_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::false_memtrack_matches_c_runtime`, `rust/src/ffi_tests.rs::list_log_free_row_names_the_variable_passed_to_free`, `rust/src/ffi_tests.rs::failing_free_keeps_its_list_log_row_like_develop`, `rust/src/output/legacy_files.rs::nondet_rows_use_develop_types_and_value_formats` **Rust:** `heap_log.csv`, `allocation_log.csv`, `list_log.csv` and `klee_log.csv` are rebuilt at the end of the run by replaying the memory and nondet records in order with the rules of `AnalysisModeMemtrack.c`, including develop's own step counter. For the calls of -`tests/legacy_golden/drivers/` the bytes match develop. These cases -differ: - -- **Pointer variable of a free.** `map2check_map_free` receives the block, - not the pointer variable, so the FREE row of `list_log.csv` names the - variable of the newest row that points to the block. Develop names the - variable passed to `map2check_free`. -- **Failing `free`.** Develop's `map2check_free` appends that FREE row - before it checks (`AnalysisModeMemtrack.c:346-358`). Here - `map2check_check_free` aborts before `map2check_map_free` records it, so - the row is missing from `list_log.csv`. +`tests/legacy_golden/drivers/` the bytes match develop. Since Task 2.5 +`map2check_check_free` and `map2check_map_free` receive the pointer +variable's slot, as develop's `map2check_free` does, so the FREE row names +the variable that was freed, and a failing free keeps the row develop +appends before it checks (`AnalysisModeMemtrack.c:346-358`); the golden +`FALSE-FREE` covers that path. These cases still differ: + - **`klee_log.csv`.** Written on every violating witness run. Develop writes it only with the AFL++ or KLEE generator; `NonDetGeneratorNone.c` writes none (`:42`). A `float` value, which develop has no logger for, is @@ -547,13 +543,9 @@ differ: **Reproduce:** ```c -map2check_set_mode(AnalysisMode_Memtrack); +map2check_set_mode(AnalysisMode_Overflow); map2check_enable_witness(); map2check_init(); -map2check_map_malloc((void *)0x3000, 8); -map2check_map_store_pointer((void *)0x10, (void *)0x3000, 0, "p", 1, "main"); -map2check_map_store_pointer((void *)0x20, (void *)0x3000, 0, "q", 2, "main"); -map2check_check_free("p", (void *)0x3000, 0, 3, "main"); -map2check_map_free("p", (void *)0x3000, 0, 3, "main"); -/* list_log.csv: the FREE row names 0x20 (q); develop names 0x10 (p) */ +map2check_binop_add_signed(INT64_MAX, 1, 64, 1, "main"); +/* klee_log.csv is written; develop's NonDetGeneratorNone.c writes none */ ``` diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index de89369..21d0838 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -256,6 +256,7 @@ pub extern "C" fn vcc_reset_meta_data() { s.result.ok = true; s.result.line = 0; s.result.function_name.clear(); + s.failed_free = None; Ok(()) }) { log_error(&e); @@ -879,11 +880,22 @@ pub unsafe extern "C" fn map2check_map_posix_memalign( ); } +/// Records the release of the heap block `ptr_address` by `free(p)`, where +/// `var_address` is the address of the pointer variable `p` whose value the +/// program freed (develop's `map2check_free` receives that slot, +/// `MemoryTrackPass.cpp:818`). The release is keyed on the block; the slot +/// only names the variable in the legacy `list_log.csv` +/// (`AnalysisModeMemtrack.c:339-347`). The pass loads `*var_address` for +/// `ptr_address`, calls `map2check_check_free` first with the same +/// arguments, and calls neither for a NULL block. +/// /// # Safety /// `var_name` and `function_name` must be null or valid null-terminated C strings. +/// `var_address` is only stored, never dereferenced. #[no_mangle] pub unsafe extern "C" fn map2check_map_free( var_name: *const std::os::raw::c_char, + var_address: *const std::os::raw::c_void, ptr_address: *const std::os::raw::c_void, scope: u32, line_number: u32, @@ -912,7 +924,9 @@ pub unsafe extern "C" fn map2check_map_free( false, ); entry.set_free(); - s.memtrack.push(entry.with_origin(MemOrigin::Free)); + s.memtrack.push(entry.with_origin(MemOrigin::Free { + var_address: var_address as usize, + })); Ok(()) }) { log_error(&e); @@ -1293,15 +1307,23 @@ pub unsafe extern "C" fn map2check_check_load( ); } -/// Checks whether freeing `ptr` is valid. +/// Checks whether `free(p)` of the heap block `ptr` is valid, where +/// `var_address` is the address of the pointer variable `p` (the same +/// arguments as `map2check_map_free`, which the pass calls after it). +/// +/// On a first violation, keeps the FREE row that develop's `map2check_free` +/// appends to `list_log.csv` before it fails (`AnalysisModeMemtrack.c:346-358`); +/// the row is not a release. /// /// # Safety -/// `function_name` must be null or a valid null-terminated C string; `ptr` may be null. +/// `name` and `function_name` must be null or valid null-terminated C strings; +/// `ptr` and `var_address` may be null and are never dereferenced. #[no_mangle] pub unsafe extern "C" fn map2check_check_free( - _name: *const std::os::raw::c_char, + name: *const std::os::raw::c_char, + var_address: *const std::os::raw::c_void, ptr: *const std::os::raw::c_void, - _scope: u32, + scope: u32, line: u32, function_name: *const std::os::raw::c_char, ) { @@ -1310,17 +1332,37 @@ pub unsafe extern "C" fn map2check_check_free( if !check_active(Check::Free) { return; } - // SAFETY: caller guarantees function_name is valid or null. + // SAFETY: caller guarantees string pointers are valid or null. let fname = unsafe { c_str(function_name) }; let ctx = VccContext::new(line, 0, fname); - run_vcc_check( - FreeChecker { - address: ptr as usize, - }, - &ctx, - line, - fname, - ); + let checker = FreeChecker { + address: ptr as usize, + }; + match checker.check(&ctx) { + Ok(crate::analysismode::VccOutcome::Violated { property }) => { + // SAFETY: as above. + let var_name = unsafe { c_str(name) }; + if let Err(e) = state::with_state(|s| { + if s.result.ok { + let addr = ptr as usize; + let mut entry = MemTrackEntry::new( + 0, line, scope, addr, addr, true, false, var_name, fname, 0, 1, + false, + ); + entry.set_free(); + s.failed_free = Some(entry.with_origin(MemOrigin::Free { + var_address: var_address as usize, + })); + } + Ok(()) + }) { + log_error(&e); + } + record_violation(property, line, fname, false); + } + Ok(crate::analysismode::VccOutcome::Safe) => {} + Err(e) => log_error(&e), + } }, (), ); diff --git a/rust/src/ffi_golden_tests.rs b/rust/src/ffi_golden_tests.rs index bd6a61f..34f4d21 100644 --- a/rust/src/ffi_golden_tests.rs +++ b/rust/src/ffi_golden_tests.rs @@ -204,15 +204,34 @@ fn memory() { } } -/// `free(p)` as the pass emits it: the check, then the release. -fn free_p() { +/// `free(p)` at `line` as the pass emits it for a free of a load of `p`: +/// the check, then the release, both with the slot of `p` and the block. +fn free_p_at(line: u32) { // SAFETY: as in `memory`. unsafe { - map2check_check_free(c"p".as_ptr(), addr(BLOCK), 1, 12, c"main".as_ptr()); - map2check_map_free(c"p".as_ptr(), addr(BLOCK), 1, 12, c"main".as_ptr()); + map2check_check_free( + c"p".as_ptr(), + addr(SLOT_P), + addr(BLOCK), + 1, + line, + c"main".as_ptr(), + ); + map2check_map_free( + c"p".as_ptr(), + addr(SLOT_P), + addr(BLOCK), + 1, + line, + c"main".as_ptr(), + ); } } +fn free_p() { + free_p_at(12); +} + /// Starts the analysis as the instrumented witness binary does. fn start(mode: AnalysisMode) { println!("{STARTED}"); @@ -228,8 +247,9 @@ fn false_free_child() { start(AnalysisMode::Memtrack); memory(); free_p(); - // SAFETY: as in `memory`. - unsafe { map2check_check_free_resolved_address(addr(BLOCK), 13, c"main".as_ptr(), 0) }; + // The second free fails in the check, after which develop's + // `map2check_free` has already appended its FREE row. + free_p_at(13); map2check_success(); } } @@ -239,6 +259,23 @@ fn false_free_matches_c_runtime() { assert_matches_golden("FALSE-FREE", "false_free_child"); } +#[test] +fn false_free_resolved_child() { + if is_child() { + start(AnalysisMode::Memtrack); + memory(); + free_p(); + // SAFETY: as in `memory`. + unsafe { map2check_check_free_resolved_address(addr(BLOCK), 13, c"main".as_ptr(), 0) }; + map2check_success(); + } +} + +#[test] +fn false_free_resolved_matches_c_runtime() { + assert_matches_golden("FALSE-FREE-RESOLVED", "false_free_resolved_child"); +} + #[test] fn false_deref_child() { if is_child() { diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 2ea821f..2039cb2 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -553,11 +553,174 @@ fn map_calloc_ignores_negative_arguments() { assert!(snapshot(|s| s.memtrack.is_empty())); } +/// `list_log.csv` as develop would write it for the records so far. +fn list_log_rows() -> Vec { + let dir = tempfile::tempdir().unwrap(); + snapshot(|s| { + crate::output::legacy_files::LegacyLogs::replay( + &s.memtrack, + &s.nondets, + s.failed_free.as_ref(), + false, + ) + .write_list_log(dir.path()) + }) + .unwrap(); + std::fs::read_to_string(dir.path().join("list_log.csv")) + .unwrap() + .lines() + .map(str::to_owned) + .collect() +} + +#[test] +fn list_log_free_row_names_the_variable_passed_to_free() { + // develop `map2check_free` (AnalysisModeMemtrack.c:339-347) takes the + // pointer variable's slot, so its FREE row names that variable even + // when another variable holds the same block (SD-17 before Task 2.5). + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x3000), 8); + map2check_map_store_pointer( + addr(0x10), + addr(0x3000), + 0, + c"p".as_ptr(), + 1, + c"main".as_ptr(), + ); + map2check_map_store_pointer( + addr(0x20), + addr(0x3000), + 0, + c"q".as_ptr(), + 2, + c"main".as_ptr(), + ); + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x3000), + 0, + 3, + c"main".as_ptr(), + ); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x3000), + 0, + 3, + c"main".as_ptr(), + ); + } + assert_ok(); + let rows = list_log_rows(); + assert_eq!(rows[2], "0;0x10;0x3000;0;1;0;p;3;main;2", "{rows:#?}"); +} + +#[test] +fn failing_free_keeps_its_list_log_row_like_develop() { + // develop appends the FREE row before `is_invalid_free` decides + // (AnalysisModeMemtrack.c:346-358); the check here runs first, so it + // keeps the row for the legacy logs. + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_map_malloc(addr(0x3000), 8); + map2check_map_store_pointer( + addr(0x10), + addr(0x3000), + 0, + c"p".as_ptr(), + 1, + c"main".as_ptr(), + ); + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x3000), + 0, + 2, + c"main".as_ptr(), + ); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x3000), + 0, + 2, + c"main".as_ptr(), + ); + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x3000), + 0, + 3, + c"main".as_ptr(), + ); + } + assert_violated(ViolatedProperty::MemsafetyFree, 3, "main"); + let rows = list_log_rows(); + assert_eq!(rows.len(), 3, "{rows:#?}"); + assert_eq!(rows[2], "0;0x10;0x3000;0;1;0;p;3;main;2"); + // Not a release: the memory records are unchanged by the failed free. + assert_eq!(snapshot(|s| s.memtrack.len()), 3); +} + +#[test] +fn vcc_reset_meta_data_drops_the_failing_free_row() { + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x3000), + 0, + 3, + c"main".as_ptr(), + ) + }; + assert!(snapshot(|s| s.failed_free.is_some())); + vcc_reset_meta_data(); + assert!(snapshot(|s| s.failed_free.is_none())); +} + +#[test] +fn only_the_first_violation_keeps_a_failing_free_row() { + let _guard = fresh(); + // SAFETY: C-string literals are valid; fake addresses are never dereferenced. + unsafe { + map2check_binop_add_signed(i64::MAX, 1, 64, 1, c"main".as_ptr()); + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x3000), + 0, + 3, + c"main".as_ptr(), + ); + } + assert!(snapshot(|s| s.failed_free.is_none())); +} + #[test] fn map_free_records_released_slot() { let _guard = fresh(); // SAFETY: C-string literals are valid; fake address is never dereferenced. - unsafe { map2check_map_free(c"p".as_ptr(), addr(0x1000), 1, 9, c"main".as_ptr()) }; + unsafe { + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x1000), + 1, + 9, + c"main".as_ptr(), + ) + }; let e = only_memtrack(); assert_eq!( (e.line, e.scope, e.var_addr, e.size_destiny), @@ -976,9 +1139,23 @@ fn sd03_safe_check_keeps_earlier_violation() { // SAFETY: as above. unsafe { map2check_check_deref(addr(0x1000), 0, 3, c"main".as_ptr()); - map2check_check_free(c"p".as_ptr(), addr(0x1000), 0, 4, c"main".as_ptr()); + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x1000), + 0, + 4, + c"main".as_ptr(), + ); map2check_check_free_resolved_address(addr(0x1000), 5, c"main".as_ptr(), 0); - map2check_map_free(c"p".as_ptr(), addr(0x1000), 0, 5, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x1000), + 0, + 5, + c"main".as_ptr(), + ); } map2check_check_mem_endprog(); map2check_set_memcleanup(); @@ -994,7 +1171,14 @@ fn sd03_later_violation_does_not_overwrite_first() { // SAFETY: C-string literals are valid; fake addresses are never dereferenced. unsafe { map2check_check_load(addr(0x9000), 7, 0, 4, c"first".as_ptr()); - map2check_check_free(c"p".as_ptr(), addr(0x9000), 0, 8, c"main".as_ptr()); + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x9000), + 0, + 8, + c"main".as_ptr(), + ); map2check_check_deref(std::ptr::null(), 0, 9, c"main".as_ptr()); set_false_result(5, 10, c"main".as_ptr()); map2check_binop_add_int(i32::MAX, 1, 11, 0, c"main".as_ptr()); @@ -1011,7 +1195,14 @@ fn deref_after_free_is_invalid() { // SAFETY: C-string literals are valid; fake addresses are never dereferenced. unsafe { map2check_map_malloc(addr(0x2000), 16); - map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x2000), + 0, + 9, + c"main".as_ptr(), + ); map2check_check_deref(addr(0x2000), 0, 10, c"main".as_ptr()); } assert_violated(ViolatedProperty::MemsafetyDeref, 10, "main"); @@ -1025,7 +1216,14 @@ fn load_after_free_is_invalid() { // SAFETY: C-string literals are valid; fake addresses are never dereferenced. unsafe { map2check_map_malloc(addr(0x2000), 16); - map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x2000), + 0, + 9, + c"main".as_ptr(), + ); map2check_check_load(addr(0x2004), 10, 0, 4, c"main".as_ptr()); } assert_violated(ViolatedProperty::MemsafetyDeref, 10, "main"); @@ -1039,18 +1237,39 @@ fn reallocation_at_same_address_is_valid_again() { // SAFETY: C-string literals are valid; fake addresses are never dereferenced. unsafe { map2check_map_malloc(addr(0x2000), 16); - map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x2000), + 0, + 9, + c"main".as_ptr(), + ); map2check_map_malloc(addr(0x2000), 16); map2check_check_load(addr(0x2004), 10, 0, 4, c"main".as_ptr()); map2check_check_deref(addr(0x2000), 0, 11, c"main".as_ptr()); - map2check_check_free(c"p".as_ptr(), addr(0x2000), 0, 12, c"main".as_ptr()); + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x2000), + 0, + 12, + c"main".as_ptr(), + ); map2check_check_free_resolved_address(addr(0x2000), 13, c"main".as_ptr(), 0); } assert_ok(); // SAFETY: as above. unsafe { - map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 14, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x2000), + 0, + 14, + c"main".as_ptr(), + ); map2check_check_load(addr(0x2004), 15, 0, 4, c"main".as_ptr()); } assert_violated(ViolatedProperty::MemsafetyDeref, 15, "main"); @@ -1062,7 +1281,14 @@ fn reallocation_with_smaller_size_shrinks_the_valid_range() { // SAFETY: C-string literals are valid; fake addresses are never dereferenced. unsafe { map2check_map_malloc(addr(0x2000), 16); - map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x2000), + 0, + 9, + c"main".as_ptr(), + ); map2check_map_malloc(addr(0x2000), 8); map2check_check_load(addr(0x2004), 10, 0, 4, c"main".as_ptr()); } @@ -1101,7 +1327,14 @@ fn pointer_store_does_not_revive_a_freed_block() { // SAFETY: C-string literals are valid; fake addresses are never dereferenced. unsafe { map2check_map_malloc(addr(0x2000), 16); - map2check_map_free(c"p".as_ptr(), addr(0x2000), 0, 9, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x2000), + 0, + 9, + c"main".as_ptr(), + ); map2check_map_store_pointer( addr(0x3000), addr(0x2000), @@ -1121,11 +1354,27 @@ fn check_free_accepts_live_allocation_and_null() { // SAFETY: C-string literals are valid; fake addresses are never dereferenced. unsafe { map2check_map_malloc(addr(0x1000), 16); - map2check_check_free(c"p".as_ptr(), addr(0x1000), 0, 20, c"main".as_ptr()); + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x1000), + 0, + 20, + c"main".as_ptr(), + ); } assert_ok(); // SAFETY: as above. - unsafe { map2check_check_free(c"p".as_ptr(), std::ptr::null(), 0, 20, c"main".as_ptr()) }; + unsafe { + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + std::ptr::null(), + 0, + 20, + c"main".as_ptr(), + ) + }; assert_ok(); } @@ -1135,14 +1384,37 @@ fn check_free_rejects_double_free_and_untracked_address() { // SAFETY: C-string literals are valid; fake addresses are never dereferenced. unsafe { map2check_map_malloc(addr(0x1000), 16); - map2check_map_free(c"p".as_ptr(), addr(0x1000), 0, 19, c"main".as_ptr()); - map2check_check_free(c"p".as_ptr(), addr(0x1000), 0, 20, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x1000), + 0, + 19, + c"main".as_ptr(), + ); + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x1000), + 0, + 20, + c"main".as_ptr(), + ); } assert_violated(ViolatedProperty::MemsafetyFree, 20, "main"); state::reset(); // SAFETY: as above. - unsafe { map2check_check_free(c"p".as_ptr(), addr(0x9000), 0, 21, c"main".as_ptr()) }; + unsafe { + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x9000), + 0, + 21, + c"main".as_ptr(), + ) + }; assert_violated(ViolatedProperty::MemsafetyFree, 21, "main"); } @@ -1158,7 +1430,14 @@ fn check_free_resolved_address_rejects_freed_target() { // SAFETY: as above. unsafe { - map2check_map_free(c"p".as_ptr(), addr(0x1000), 0, 30, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x1000), + 0, + 30, + c"main".as_ptr(), + ); map2check_check_free_resolved_address(addr(0x1000), 31, c"main".as_ptr(), 0); } assert_violated(ViolatedProperty::MemsafetyFree, 31, "main"); @@ -1175,7 +1454,16 @@ fn sd06_untracked_free_is_invalid() { state::reset(); // SAFETY: as above. - unsafe { map2check_check_free(c"p".as_ptr(), addr(0x5000), 0, 4, c"main".as_ptr()) }; + unsafe { + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x5000), + 0, + 4, + c"main".as_ptr(), + ) + }; assert_violated(ViolatedProperty::MemsafetyFree, 4, "main"); } @@ -1247,7 +1535,14 @@ fn sd13_free_of_a_never_stored_pointer_is_valid() { map2check_map_malloc(addr(0x1000), 8); map2check_check_free_resolved_address(addr(0x1000), 3, c"main".as_ptr(), 0); map2check_map_malloc(addr(0x2000), 8); - map2check_check_free(c"p".as_ptr(), addr(0x2000), 0, 4, c"main".as_ptr()); + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0x2000), + 0, + 4, + c"main".as_ptr(), + ); } assert_ok(); } @@ -1272,7 +1567,14 @@ fn check_deref_accepts_live_and_rejects_freed_null_or_untracked() { // SAFETY: as above. unsafe { - map2check_map_free(c"p".as_ptr(), addr(0x1000), 0, 41, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x1000), + 0, + 41, + c"main".as_ptr(), + ); map2check_check_deref(addr(0x1000), 0, 42, c"main".as_ptr()); } assert_violated(ViolatedProperty::MemsafetyDeref, 42, "main"); @@ -1312,7 +1614,14 @@ fn check_mem_endprog_accepts_freed_allocation() { // SAFETY: C-string literals are valid; fake address is never dereferenced. unsafe { map2check_map_malloc(addr(0x1000), 16); - map2check_map_free(c"p".as_ptr(), addr(0x1000), 0, 50, c"main".as_ptr()); + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x1000), + 0, + 50, + c"main".as_ptr(), + ); } map2check_check_mem_endprog(); assert_ok(); @@ -1524,7 +1833,16 @@ fn reports_in(mode: AnalysisMode) -> [ViolatedProperty; 7] { // are never dereferenced. &|| unsafe { map2check_check_load(addr(0xdead), 1, 0, 4, c"main".as_ptr()) }, &|| unsafe { map2check_check_deref(addr(0xdead), 0, 1, c"main".as_ptr()) }, - &|| unsafe { map2check_check_free(c"p".as_ptr(), addr(0xdead), 0, 1, c"main".as_ptr()) }, + &|| unsafe { + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0xdead), + 0, + 1, + c"main".as_ptr(), + ) + }, &|| unsafe { map2check_check_free_resolved_address(addr(0xdead), 1, c"main".as_ptr(), 0) }, &|| unsafe { map2check_binop_add_int(i32::MAX, 1, 1, 0, c"main".as_ptr()) }, &|| unsafe { map2check_is_valid_assert(1, c"main".as_ptr(), 0) }, @@ -1637,7 +1955,16 @@ fn unset_mode_keeps_every_check_active() { // SAFETY (all closures): as in `reports_in`. &|| unsafe { map2check_check_load(addr(0xdead), 1, 0, 4, c"main".as_ptr()) }, &|| unsafe { map2check_check_deref(addr(0xdead), 0, 1, c"main".as_ptr()) }, - &|| unsafe { map2check_check_free(c"p".as_ptr(), addr(0xdead), 0, 1, c"main".as_ptr()) }, + &|| unsafe { + map2check_check_free( + c"p".as_ptr(), + addr(0x10), + addr(0xdead), + 0, + 1, + c"main".as_ptr(), + ) + }, &|| unsafe { map2check_check_free_resolved_address(addr(0xdead), 1, c"main".as_ptr(), 0) }, &|| unsafe { map2check_binop_add_int(i32::MAX, 1, 1, 0, c"main".as_ptr()) }, &|| unsafe { map2check_is_valid_assert(1, c"main".as_ptr(), 0) }, @@ -1907,7 +2234,16 @@ fn memcleanup_mode_freed_block_is_no_leak() { malloc_at(0x3000); store(0x10, 0x3000); // SAFETY: C-string literals are valid; fake address is never dereferenced. - unsafe { map2check_map_free(c"p".as_ptr(), addr(0x3000), 0, 2, c"main".as_ptr()) }; + unsafe { + map2check_map_free( + c"p".as_ptr(), + addr(0x10), + addr(0x3000), + 0, + 2, + c"main".as_ptr(), + ) + }; assert_eq!(memcleanup_endprog(), P::None); } diff --git a/rust/src/memtrack/mod.rs b/rust/src/memtrack/mod.rs index 40c1024..f47fcc5 100644 --- a/rust/src/memtrack/mod.rs +++ b/rust/src/memtrack/mod.rs @@ -49,8 +49,11 @@ pub enum MemOrigin { Malloc, /// `map2check_map_store_pointer`. StorePointer, - /// `map2check_map_free`. - Free, + /// `map2check_map_free`, with the address of the pointer variable that + /// was freed (develop's `map2check_free` argument). The entry itself is + /// keyed on the block, so the memory checks never see the variable; + /// only the legacy list log names it. + Free { var_address: usize }, /// The release recorded by `map2check_check_free_resolved_address`. FreeResolved, } @@ -226,6 +229,9 @@ impl DevelopLogs { if e.is_dynamic && !e.is_free { logs.malloc(e.points_to); } else if e.is_free { + if let MemOrigin::Free { var_address } = e.origin { + logs.free_row(var_address, e.points_to); + } logs.free(e.points_to); } else if e.size_destiny == 0 { logs.store_pointer(e.var_addr, e.points_to); @@ -249,11 +255,21 @@ impl DevelopLogs { } } - /// The free paths, `map2check_free` (`:329-356`) and - /// `map2check_free_resolved_address` (`:160-179`): mark the row free and - /// run `update_reference_list_log(addr, FREE)`. `map2check_free` also - /// appends a FREE row for the pointer variable (`:336-341`); the Rust free - /// record carries only the heap address, so that row is not rebuilt. + /// The FREE row `map2check_free` appends for the pointer variable before + /// it releases the block (`:336-343`). A NULL block returns before it. + fn free_row(&mut self, var: usize, addr: usize) { + if addr != 0 { + self.list.push(ListRow { + var, + points_to: addr, + status: AddressStatus::Free, + }); + } + } + + /// The free paths, `map2check_free` (`:329-356`, after [`Self::free_row`]) + /// and `map2check_free_resolved_address` (`:160-179`): mark the row free + /// and run `update_reference_list_log(addr, FREE)`. fn free(&mut self, addr: usize) { if addr == 0 { return; @@ -429,6 +445,20 @@ mod tests { MemTrackEntry::new(1, 1, 0, var, value, false, false, "p", "main", 0, 1, false) } + #[test] + fn map_free_row_names_its_variable_in_the_memcleanup_verdict() { + // develop's map2check_free appends a FREE row for the variable it + // received (AnalysisModeMemcleanup.c:336-343), even when no store + // of that variable was tracked; the row still points to the block, + // so a block reallocated at the same address counts as referenced. + let freed_through_p = free(0x3000).with_origin(MemOrigin::Free { var_address: 0x10 }); + let entries = vec![alloc(0x3000), freed_through_p, alloc(0x3000)]; + assert!(has_develop_memcleanup_error(&entries)); + // A release without a variable (free_resolved) appends no row. + let entries = vec![alloc(0x3000), free(0x3000), alloc(0x3000)]; + assert!(!has_develop_memcleanup_error(&entries)); + } + #[test] fn last_leaked_address_is_the_last_live_block_in_allocation_order() { assert_eq!(last_leaked_address(&[]), None); diff --git a/rust/src/output/legacy_files.rs b/rust/src/output/legacy_files.rs index 4ef6c9c..3f83745 100644 --- a/rust/src/output/legacy_files.rs +++ b/rust/src/output/legacy_files.rs @@ -275,12 +275,15 @@ enum Event<'a> { impl LegacyLogs { /// Replays the memory and nondet records in the order the program made - /// them (both carry this library's step). `end_of_program` adds what - /// `valid_allocation_log` appends when the end-of-program verdict runs - /// (`AllocationLog.c:69-101`). + /// them (both carry this library's step). `failed_free`, the free that + /// ended the run, adds the FREE row develop's `map2check_free` appends + /// before it fails (`AnalysisModeMemtrack.c:346-358`). `end_of_program` + /// adds what `valid_allocation_log` appends when the end-of-program + /// verdict runs (`AllocationLog.c:69-101`). pub fn replay( memtrack: &[MemTrackEntry], nondets: &[NonDetEntry], + failed_free: Option<&MemTrackEntry>, end_of_program: bool, ) -> Self { let mut events: Vec<(u64, Event<'_>)> = memtrack @@ -297,6 +300,9 @@ impl LegacyLogs { Event::Nondet(e) => logs.nondet(e), } } + if let Some(e) = failed_free { + logs.free_row(e); + } if end_of_program { logs.valid_allocation_log(); } @@ -332,7 +338,7 @@ impl LegacyLogs { }), MemOrigin::Malloc => self.malloc(e.points_to, e.size_destiny as u32), MemOrigin::StorePointer => self.store_pointer(e), - MemOrigin::Free => self.free(e), + MemOrigin::Free { .. } => self.free(e), MemOrigin::FreeResolved => self.free_resolved(e.points_to, e.line), MemOrigin::Unspecified => {} } @@ -410,21 +416,26 @@ impl LegacyLogs { } /// `map2check_free` (`AnalysisModeMemtrack.c:339-366`): a FREE row for - /// the pointer variable, then the block is released. The C runtime reads - /// the variable's address from its argument; this library's free record - /// has only the block, so the variable is the one of the newest list row - /// that points to the block (SD-17). + /// the pointer variable, then the block is released. fn free(&mut self, e: &MemTrackEntry) { + if self.free_row(e) { + self.release(e.points_to, e.line); + } + } + + /// The FREE row `map2check_free` appends for the pointer variable whose + /// slot it received (`:339-347`); returns false for a NULL block, where + /// develop returns first (`:342-344`). A record without the variable + /// (built outside `map2check_map_free`) names the block, as before. + fn free_row(&mut self, e: &MemTrackEntry) -> bool { let addr = e.points_to; if addr == 0 { - return; + return false; } - let var = self - .list - .iter() - .rev() - .find(|r| r.points_to == addr) - .map_or(addr, |r| r.var); + let var = match e.origin { + MemOrigin::Free { var_address } => var_address, + _ => addr, + }; let step = self.next_step(); self.list.push(ListRow { var, @@ -437,7 +448,7 @@ impl LegacyLogs { function: e.function_name.clone(), step, }); - self.release(addr, e.line); + true } /// `map2check_free_resolved_address` (`AnalysisModeMemtrack.c:164-183`). diff --git a/rust/src/output/termination.rs b/rust/src/output/termination.rs index f9f4b99..b52db37 100644 --- a/rust/src/output/termination.rs +++ b/rust/src/output/termination.rs @@ -69,7 +69,12 @@ fn write_aux_files(dir: &Path, config: Config, s: &AnalysisState, end_of_program legacy_files::TRACKBB_LOG_FILE, legacy_files::write_trackbb_log(dir, &s.bbtrack), ); - let logs = LegacyLogs::replay(&s.memtrack, &s.nondets, end_of_program); + let logs = LegacyLogs::replay( + &s.memtrack, + &s.nondets, + s.failed_free.as_ref(), + end_of_program, + ); if matches!( config.mode, Some(AnalysisMode::Memtrack | AnalysisMode::Memcleanup) diff --git a/rust/src/state.rs b/rust/src/state.rs index 77afdee..e564573 100644 --- a/rust/src/state.rs +++ b/rust/src/state.rs @@ -82,6 +82,11 @@ pub struct AnalysisState { pub bbtrack: Vec, pub legacy_nondet_logs: Vec>, pub legacy_bbtrack_logs: Vec>, + /// The free that `map2check_check_free` found invalid, when it is the + /// first violation. Develop's `map2check_free` appends its FREE row to + /// the list log before it checks (`AnalysisModeMemtrack.c:346-358`), so + /// the legacy logs keep it; it is not a release and no check reads it. + pub failed_free: Option, } impl AnalysisState { diff --git a/test/codeAPI_by_caller_test.c b/test/codeAPI_by_caller_test.c index 7b8fb17..471bc25 100644 --- a/test/codeAPI_by_caller_test.c +++ b/test/codeAPI_by_caller_test.c @@ -34,15 +34,16 @@ int main(void) { /* Heap allocation: check-before-free must be safe on both free checks. */ void *heap_ptr = (void *)0x1000; /* synthetic address, never dereferenced */ + void *heap_var = &heap_ptr; /* the pointer variable being freed */ map2check_map_malloc(heap_ptr, 16); - map2check_check_free("heap_obj", heap_ptr, 0, 15, "main"); + map2check_check_free("heap_obj", heap_var, heap_ptr, 0, 15, "main"); assert(get_current_property() == None); map2check_check_free_resolved_address(heap_ptr, 15, "main", 0); assert(get_current_property() == None); /* After the free is recorded, both checks must flag the double free. */ - map2check_map_free("heap_obj", heap_ptr, 0, 16, "main"); - map2check_check_free("heap_obj", heap_ptr, 0, 17, "main"); + map2check_map_free("heap_obj", heap_var, heap_ptr, 0, 16, "main"); + map2check_check_free("heap_obj", heap_var, heap_ptr, 0, 17, "main"); assert(get_current_property() == MemsafetyFree); map2check_check_free_resolved_address(heap_ptr, 17, "main", 0); assert(get_current_property() == MemsafetyFree); diff --git a/tests/compat_docs.rs b/tests/compat_docs.rs index f7545cb..9c6b770 100644 --- a/tests/compat_docs.rs +++ b/tests/compat_docs.rs @@ -284,7 +284,14 @@ fn integration_guide_json_examples_match_the_serializer() { // SAFETY: C-string literals are valid; fake addresses are never dereferenced. unsafe { map2check_map_malloc(0x1000 as *const _, 16); - map2check_check_free(c"p".as_ptr(), 0x2000 as *const _, 0, 20, c"main".as_ptr()); + map2check_check_free( + c"p".as_ptr(), + 0x10 as *const _, + 0x2000 as *const _, + 0, + 20, + c"main".as_ptr(), + ); } assert_eq!( report(), diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index 77ad14e..6c7d10c 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -72,9 +72,9 @@ static void run_pre_init_boundary(void) { map2check_map_store_pointer(&stack_obj, heap_obj, 0, "", 1, "main"); map2check_check_load(heap_obj, 1, 0, 0, "main"); map2check_check_deref(heap_obj, 0, 1, "main"); - map2check_check_free("", heap_obj, 0, 1, "main"); + map2check_check_free("", &stack_obj, heap_obj, 0, 1, "main"); map2check_check_free_resolved_address(heap_obj, 1, "main", 0); - map2check_map_free("", heap_obj, 0, 1, "main"); + map2check_map_free("", &stack_obj, heap_obj, 0, 1, "main"); map2check_check_mem_endprog(); (void)map2check_save_basic_block_log(1, "main"); @@ -161,7 +161,7 @@ static void run_bad_inputs(void) { /* memory/deref/free/load checks, pre-init */ map2check_check_load(NULL, 1, 0, 8, NULL); map2check_check_deref(NULL, 0, 2, NULL); - map2check_check_free(NULL, NULL, 0, 3, NULL); + map2check_check_free(NULL, NULL, NULL, 0, 3, NULL); map2check_check_free_resolved_address(NULL, 3, NULL, 1); map2check_check_mem_endprog(); map2check_reach_target(NULL, -1, -1); @@ -179,7 +179,7 @@ static void run_bad_inputs(void) { map2check_map_posix_memalign(&null_block, -1); } map2check_map_store_pointer(NULL, NULL, 0, NULL, 6, NULL); - map2check_map_free(NULL, NULL, 0, 7, NULL); + map2check_map_free(NULL, NULL, NULL, 0, 7, NULL); /* basic-block tracking, pre-init */ map2check_save_basic_block_log(8, NULL); @@ -265,11 +265,11 @@ static void run_valid_pass(void) { free(aligned); } } - map2check_map_free("buf", (void *)0x1, 1, 22, "main"); + map2check_map_free("buf", (void *)0x1, (void *)0x2, 1, 22, "main"); map2check_check_load((void *)0x1, 23, 1, 4, "main"); map2check_check_deref((void *)0x1, 1, 24, "main"); - map2check_check_free("buf", (void *)0x1, 1, 25, "main"); + map2check_check_free("buf", (void *)0x1, (void *)0x2, 1, 25, "main"); map2check_check_free_resolved_address(NULL, 25, "main", 1); map2check_check_mem_endprog(); @@ -380,7 +380,7 @@ static int test_result_and_control_flags(void) { * later violation leave it in place. */ map2check_reset(); set_false_result(Overflow, 10, "main"); - map2check_check_free(NULL, NULL, 0, 11, NULL); + map2check_check_free(NULL, NULL, NULL, 0, 11, NULL); set_false_result(Reachability, 12, "main"); if (get_current_property() != Overflow) { return 5; diff --git a/tests/legacy_golden/FALSE-FREE-RESOLVED/.exit_status b/tests/legacy_golden/FALSE-FREE-RESOLVED/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE-RESOLVED/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/FALSE-FREE-RESOLVED/allocation_log.csv b/tests/legacy_golden/FALSE-FREE-RESOLVED/allocation_log.csv new file mode 100644 index 0000000..1d1e507 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE-RESOLVED/allocation_log.csv @@ -0,0 +1 @@ +0x10000100;0;1 diff --git a/tests/legacy_golden/FALSE-FREE-RESOLVED/heap_log.csv b/tests/legacy_golden/FALSE-FREE-RESOLVED/heap_log.csv new file mode 100644 index 0000000..b4ac048 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE-RESOLVED/heap_log.csv @@ -0,0 +1,2 @@ +0x10000000;p;8 +0x10000040;main;1 diff --git a/tests/legacy_golden/FALSE-FREE-RESOLVED/klee_log.csv b/tests/legacy_golden/FALSE-FREE-RESOLVED/klee_log.csv new file mode 100644 index 0000000..dadd065 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE-RESOLVED/klee_log.csv @@ -0,0 +1,5 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;-3;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/FALSE-FREE-RESOLVED/list_log.csv b/tests/legacy_golden/FALSE-FREE-RESOLVED/list_log.csv new file mode 100644 index 0000000..645df4d --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE-RESOLVED/list_log.csv @@ -0,0 +1,2 @@ +0;0x10000000;0x10000100;1;0;1;p;11;main;5 +0;0x10000000;0x10000100;1;1;0;p;12;main;6 diff --git a/tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_checked_error b/tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_property b/tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_property new file mode 100644 index 0000000..138db76 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_property @@ -0,0 +1,3 @@ +FALSE-FREE +Line: 13 +Function: main diff --git a/tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_property_free b/tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_property_free new file mode 100644 index 0000000..138db76 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE-RESOLVED/map2check_property_free @@ -0,0 +1,3 @@ +FALSE-FREE +Line: 13 +Function: main diff --git a/tests/legacy_golden/FALSE-FREE-RESOLVED/track_bb_log.st b/tests/legacy_golden/FALSE-FREE-RESOLVED/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/FALSE-FREE-RESOLVED/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/FALSE-FREE/list_log.csv b/tests/legacy_golden/FALSE-FREE/list_log.csv index 645df4d..ee49de3 100644 --- a/tests/legacy_golden/FALSE-FREE/list_log.csv +++ b/tests/legacy_golden/FALSE-FREE/list_log.csv @@ -1,2 +1,3 @@ 0;0x10000000;0x10000100;1;0;1;p;11;main;5 0;0x10000000;0x10000100;1;1;0;p;12;main;6 +0;0x10000000;0x10000100;1;1;0;p;13;main;7 diff --git a/tests/legacy_golden/README.md b/tests/legacy_golden/README.md index 386afdb..e17e3b2 100644 --- a/tests/legacy_golden/README.md +++ b/tests/legacy_golden/README.md @@ -20,7 +20,8 @@ compiled with `clang-16`. Each binary runs in a fresh directory. | Tag | Mode file | Calls after `map2check_init(0)` and the shared tracking calls | |---|---|---| -| `FALSE-FREE` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_free` of `p`, `map2check_free_resolved_address` of the same block | +| `FALSE-FREE` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_free` of `p` twice (the second fails after appending its FREE row) | +| `FALSE-FREE-RESOLVED` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_free` of `p`, `map2check_free_resolved_address` of the same block | | `FALSE-DEREF` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_load` of an unallocated address, `map2check_check_deref` | | `FALSE-MEMTRACK` | `AnalysisModeMemtrack.c` | memory prelude, `map2check_success` (the block leaks) | | `FALSE-MEMCLEANUP` | `AnalysisModeMemcleanup.c` | memory prelude, `map2check_success` (the block leaks, `p` points to it) | diff --git a/tests/legacy_golden/drivers/FALSE-FREE-RESOLVED.c b/tests/legacy_golden/drivers/FALSE-FREE-RESOLVED.c new file mode 100644 index 0000000..7b0f94f --- /dev/null +++ b/tests/legacy_golden/drivers/FALSE-FREE-RESOLVED.c @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Memtrack mode: free(p), then a second free of the same block through + * map2check_free_resolved_address (a free of a value that is not a load). + */ +#include "golden.h" + +int __map2check_main__(void) { + golden_arena(); + map2check_init(0); + golden_tracking(); + golden_memory(); + map2check_free("p", SLOT_P, 1, 12, "main"); + map2check_free_resolved_address(BLOCK, 13, "main", 0); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/FALSE-FREE.c b/tests/legacy_golden/drivers/FALSE-FREE.c index cee876b..aeae90b 100644 --- a/tests/legacy_golden/drivers/FALSE-FREE.c +++ b/tests/legacy_golden/drivers/FALSE-FREE.c @@ -1,5 +1,7 @@ /* SPDX-License-Identifier: GPL-3.0-only - * Memtrack mode: free(p), then a second free of the same block. + * Memtrack mode: free(p) twice through map2check_free, the path of a free + * whose argument is a load of the pointer variable. The second call appends + * its FREE row to the list log before it fails (AnalysisModeMemtrack.c:346-358). */ #include "golden.h" @@ -9,7 +11,7 @@ int __map2check_main__(void) { golden_tracking(); golden_memory(); map2check_free("p", SLOT_P, 1, 12, "main"); - map2check_free_resolved_address(BLOCK, 13, "main", 0); + map2check_free("p", SLOT_P, 1, 13, "main"); map2check_success(); return 0; } diff --git a/tests/legacy_golden/drivers/build_goldens.sh b/tests/legacy_golden/drivers/build_goldens.sh index b1256de..4bea45b 100755 --- a/tests/legacy_golden/drivers/build_goldens.sh +++ b/tests/legacy_golden/drivers/build_goldens.sh @@ -37,6 +37,7 @@ MEMORY="AllocationLog ListLog HeapLog" # tag -> analysis mode files (:578-612) declare -A MODE=( [FALSE-FREE]="AnalysisModeMemtrack $MEMORY" + [FALSE-FREE-RESOLVED]="AnalysisModeMemtrack $MEMORY" [FALSE-DEREF]="AnalysisModeMemtrack $MEMORY" [FALSE-MEMTRACK]="AnalysisModeMemtrack $MEMORY" [FALSE-MEMCLEANUP]="AnalysisModeMemcleanup $MEMORY" @@ -49,7 +50,7 @@ declare -A MODE=( [TARGET-REACHED]="AnalysisModeNone" ) -for tag in FALSE-FREE FALSE-DEREF FALSE-MEMTRACK FALSE-MEMCLEANUP NONE \ +for tag in FALSE-FREE FALSE-FREE-RESOLVED FALSE-DEREF FALSE-MEMTRACK FALSE-MEMCLEANUP NONE \ UNKNOWN OVERFLOW DIVBYZERO POSIX-MEMALIGN ASSERT TARGET-REACHED; do bc "$DRIVERS/$tag.c" "$WORK/$tag-output.bc" inputs=("$WORK/$tag-output.bc") From 0d5b1f68ebc7ee798bc84747ba7ed95f064bca62 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 22:45:54 -0400 Subject: [PATCH 29/63] docs: refresh the symbol matrix after the core binop, posix_memalign and free exports Only the nondet rows (Task 2.5b) and the wasm rows (Task 2.8) are still missing from include/map2check.h. --- docs/cutover/symbol-matrix.tsv | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/cutover/symbol-matrix.tsv b/docs/cutover/symbol-matrix.tsv index e0633c4..f474836 100644 --- a/docs/cutover/symbol-matrix.tsv +++ b/docs/cutover/symbol-matrix.tsv @@ -4,13 +4,13 @@ map2check_alloca map2check_map_alloca 1 map2check_assert map2check_is_valid_assert 1 map2check_assume __VERIFIER_assume 1 map2check_assume_loop __VERIFIER_assume 1 -map2check_binop_add map2check_binop_add_signed 0 +map2check_binop_add map2check_binop_add_signed 1 map2check_binop_add_uint - 1 -map2check_binop_mul map2check_binop_mul_signed 0 +map2check_binop_mul map2check_binop_mul_signed 1 map2check_binop_mul_uint - 1 -map2check_binop_sdiv map2check_binop_div_signed 0 -map2check_binop_srem map2check_binop_rem_signed 0 -map2check_binop_sub map2check_binop_sub_signed 0 +map2check_binop_sdiv map2check_binop_div_signed 1 +map2check_binop_srem map2check_binop_rem_signed 1 +map2check_binop_sub map2check_binop_sub_signed 1 map2check_binop_sub_uint - 1 map2check_calloc map2check_map_calloc 1 map2check_check_deref - 1 @@ -18,7 +18,7 @@ map2check_crab_assume __VERIFIER_assume 1 map2check_free map2check_check_free+map2check_map_free 1 map2check_free_resolved_address map2check_check_free_resolved_address 1 map2check_function map2check_map_funct_address 1 -map2check_init map2check_set_mode+map2check_init 0 +map2check_init map2check_set_mode+map2check_init 1 map2check_load map2check_check_load 1 map2check_malloc map2check_map_malloc 1 map2check_non_det_bool __VERIFIER_nondet_bool 1 @@ -54,9 +54,9 @@ map2check_nondet_unsigned map2check_save_nondet_log_unsigned 0 map2check_non_det_ushort __VERIFIER_nondet_ushort 1 map2check_nondet_ushort map2check_save_nondet_log_ushort 0 map2check_non_static_alloca map2check_map_non_static_alloca 1 -map2check_posix map2check_map_posix_memalign 0 +map2check_posix map2check_map_posix_memalign 1 map2check_success map2check_success 1 -map2check_target_function map2check_reach_target 0 +map2check_target_function map2check_reach_target 1 map2check_track_bb map2check_save_basic_block_log 1 map2check_wasm_check_access map2check_wasm_check_access 0 map2check_wasm_free map2check_wasm_free 0 From 9156b093dea8d35b8a98690b7ce08bcda33ba414 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 23:16:54 -0400 Subject: [PATCH 30/63] feat: log every nondet type the pass emits as develop writes it (#6, #13) Add map2check_save_nondet_log_{unsigned,ulong,uchar,short,ushort,bool, size_t,loff_t,sector_t,pointer,pchar}, each taking a pointer to the value in the C type develop's NonDetGenerator.h gives the family, with the existing (line, scope, type_var, value, function_name) signature. klee_log.csv now matches develop for every type: the enum NONDET_TYPE number of the export and the value as NonDetPass.cpp hands it to the C int wrapper (CreateIntegerCast with isSigned = false, so narrow types are zero-extended and wide ones truncated; ptrtoint for addresses), printed with %u for UNSIGNED and %d otherwise. A char -3 is therefore logged as 253, as develop does; golden.h passed it sign-extended, which no pass emits, so the shared goldens are regenerated and only that row changes. The new golden NONDET-TYPES, captured from develop in map2check-dev:cutover-f01e8c3e, logs one value of each type. --- build.rs | 12 ++ docs/api-deltas.md | 29 ++++ docs/semantic-deltas.md | 8 +- rust/src/ffi.rs | 126 +++++++++++++++--- rust/src/ffi_golden_tests.rs | 54 ++++++++ rust/src/ffi_tests.rs | 115 ++++++++++++++++ rust/src/nondet/mod.rs | 50 +++++++ rust/src/output/legacy_files.rs | 48 +++++-- tests/legacy_golden/ASSERT/klee_log.csv | 2 +- tests/legacy_golden/DIVBYZERO/klee_log.csv | 2 +- tests/legacy_golden/FALSE-DEREF/klee_log.csv | 2 +- .../FALSE-FREE-RESOLVED/klee_log.csv | 2 +- tests/legacy_golden/FALSE-FREE/klee_log.csv | 2 +- .../FALSE-MEMCLEANUP/klee_log.csv | 2 +- .../legacy_golden/FALSE-MEMTRACK/klee_log.csv | 2 +- tests/legacy_golden/NONDET-TYPES/.exit_status | 1 + tests/legacy_golden/NONDET-TYPES/klee_log.csv | 21 +++ .../NONDET-TYPES/map2check_checked_error | 1 + .../NONDET-TYPES/map2check_property | 3 + .../NONDET-TYPES/map2check_property_assert | 3 + .../NONDET-TYPES/track_bb_log.st | 2 + tests/legacy_golden/OVERFLOW/klee_log.csv | 2 +- .../legacy_golden/POSIX-MEMALIGN/klee_log.csv | 2 +- .../legacy_golden/TARGET-REACHED/klee_log.csv | 2 +- tests/legacy_golden/UNKNOWN/klee_log.csv | 2 +- tests/legacy_golden/drivers/NONDET-TYPES.c | 70 ++++++++++ tests/legacy_golden/drivers/build_goldens.sh | 4 +- tests/legacy_golden/drivers/golden.h | 4 +- 28 files changed, 535 insertions(+), 38 deletions(-) create mode 100644 tests/legacy_golden/NONDET-TYPES/.exit_status create mode 100644 tests/legacy_golden/NONDET-TYPES/klee_log.csv create mode 100644 tests/legacy_golden/NONDET-TYPES/map2check_checked_error create mode 100644 tests/legacy_golden/NONDET-TYPES/map2check_property create mode 100644 tests/legacy_golden/NONDET-TYPES/map2check_property_assert create mode 100644 tests/legacy_golden/NONDET-TYPES/track_bb_log.st create mode 100644 tests/legacy_golden/drivers/NONDET-TYPES.c diff --git a/build.rs b/build.rs index 644c236..1b4e0c5 100644 --- a/build.rs +++ b/build.rs @@ -45,6 +45,7 @@ typedef struct _non_det_log { char str[20]; float *f; double *d; + void *p; } value; const char *function_name; } non_det_log_t; @@ -56,6 +57,17 @@ non_det_log_t *map2check_save_nondet_log_long(int line, int scope, enum var_type non_det_log_t *map2check_save_nondet_log_char(int line, int scope, enum var_type_t type_var, char *value, const char *function_name); non_det_log_t *map2check_save_nondet_log_float(int line, int scope, enum var_type_t type_var, float *value, const char *function_name); non_det_log_t *map2check_save_nondet_log_double(int line, int scope, enum var_type_t type_var, double *value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_unsigned(int line, int scope, int type_var, unsigned int *value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_ulong(int line, int scope, int type_var, unsigned long *value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_uchar(int line, int scope, int type_var, unsigned char *value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_short(int line, int scope, int type_var, short *value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_ushort(int line, int scope, int type_var, unsigned short *value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_bool(int line, int scope, int type_var, int *value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_size_t(int line, int scope, int type_var, size_t *value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_loff_t(int line, int scope, int type_var, long long *value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_sector_t(int line, int scope, int type_var, unsigned long *value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_pointer(int line, int scope, int type_var, void **value, const char *function_name); +non_det_log_t *map2check_save_nondet_log_pchar(int line, int scope, int type_var, char **value, const char *function_name); void map2check_binop_add_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name); void map2check_binop_sub_int(int32_t param1, int32_t param2, uint32_t line, uint32_t scope, const char *function_name); diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 9cbedb6..5f14c9d 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -63,6 +63,17 @@ build with `--features libfuzzer`. | `map2check_save_nondet_log_float` | `non_det_log_t *map2check_save_nondet_log_float(int line, int scope, enum var_type_t type_var, float *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_double` | `non_det_log_t *map2check_save_nondet_log_double(int line, int scope, enum var_type_t type_var, double *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_char` | `non_det_log_t *map2check_save_nondet_log_char(int line, int scope, enum var_type_t type_var, char *value, const char *function_name)` | returns `non_det_log_t *`; `char` value | same | `char` is `i8` on the Rust side; see “`char` ABI”. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_unsigned` | `non_det_log_t *map2check_save_nondet_log_unsigned(int line, int scope, int type_var, unsigned int *value, const char *function_name)` | — | new | Develop `map2check_nondet_unsigned` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UNSIGNED (5), value printed with `%u`; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_ulong` | `non_det_log_t *map2check_save_nondet_log_ulong(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_ulong` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type ULONG (6), low 32 bits with `%d`; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_uchar` | `non_det_log_t *map2check_save_nondet_log_uchar(int line, int scope, int type_var, unsigned char *value, const char *function_name)` | — | new | Develop `map2check_nondet_uchar` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UCHAR (8), value zero-extended; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_short` | `non_det_log_t *map2check_save_nondet_log_short(int line, int scope, int type_var, short *value, const char *function_name)` | — | new | Develop `map2check_nondet_short` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SHORT (14), value zero-extended as the pass does (`-2` is logged as `65534`); see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_ushort` | `non_det_log_t *map2check_save_nondet_log_ushort(int line, int scope, int type_var, unsigned short *value, const char *function_name)` | — | new | Develop `map2check_nondet_ushort` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type USHORT (3), value zero-extended; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_bool` | `non_det_log_t *map2check_save_nondet_log_bool(int line, int scope, int type_var, int *value, const char *function_name)` | — | new | Develop `map2check_nondet_bool` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type BOOL (7); develop's `bool` is `typedef int bool` (`NonDetGenerator.h:74`), so the pass widens the `_Bool` to `int` before storing it; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_size_t` | `non_det_log_t *map2check_save_nondet_log_size_t(int line, int scope, int type_var, size_t *value, const char *function_name)` | — | new | Develop `map2check_nondet_size_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SIZE_T (10), low 32 bits with `%d`; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_loff_t` | `non_det_log_t *map2check_save_nondet_log_loff_t(int line, int scope, int type_var, long long *value, const char *function_name)` | — | new | Develop `map2check_nondet_loff_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type LOFF_T (11); `loff_t` is glibc's `long long`; low 32 bits with `%d`; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_sector_t` | `non_det_log_t *map2check_save_nondet_log_sector_t(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_sector_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SECTOR_T (12); `sector_t` is `unsigned long` (`NonDetGenerator.h:112`); low 32 bits with `%d`; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_pointer` | `non_det_log_t *map2check_save_nondet_log_pointer(int line, int scope, int type_var, void **value, const char *function_name)` | — | new | Develop `map2check_nondet_pointer` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type POINTER (2); the address, logged as its low 32 bits (the pass's `ptrtoint` to `i32`); see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_pchar` | `non_det_log_t *map2check_save_nondet_log_pchar(int line, int scope, int type_var, char **value, const char *function_name)` | — | new | Develop `map2check_nondet_pchar` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type PCHAR (9); the string's address, logged as its low 32 bits; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | | `map2check_save_basic_block_log` | `bbtrack_log_t *map2check_save_basic_block_log(int32_t line, const char *function_name)` | returns `bbtrack_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_is_in_trackbb_container` | `int32_t map2check_is_in_trackbb_container(int32_t line)` | `map2check_is_in_trackbb_container` | same | | | `map2check_map_alloca` | `void map2check_map_alloca(const char *var_name, const void *ptr_address, int32_t size, int32_t size_primitive, int32_t line_number, int32_t scope)` | `container_memtracklog.h:41` | same | | @@ -150,6 +161,7 @@ Map2Check. | `map2check_crab_assume` | — | pending | #21. | | `map2check_assert` | — | pending | #21. | | `map2check_non_det_*` | 16 typed generators: bool, char, double, int, loff_t, long, pchar, pointer, sector_t, short, size_t, uchar, uint, ulong, unsigned, ushort | pending | #21; see also #6. | +| `map2check_nondet_*` | `(u32 line, u32 scope, i32 value, ptr fn)`, `double` value for `double` | pending | Core equivalents: `map2check_save_nondet_log_(line, scope, type_var, T *value, fn)` for every type; the pass stores the value in an `alloca` and passes its address (Task 3.2). See “Nondet log”. | | `map2check_nondet_cast_*` | — | pending | #21. | | `map2check_pointer_cast` | — | pending | #21. | | `map2check_wasm_malloc` | — | out-of-scope | Only used by the `feat-wasm-verification` flow (#21). | @@ -240,6 +252,23 @@ In memtrack and memcleanup modes `map2check_map_malloc` and scenario per verdict, and `rust/src/ffi_golden_tests.rs` checks that this library writes the same bytes for the same calls. +## Nondet log + +`map2check_save_nondet_log_` keeps the core signature +`(int line, int scope, int type_var, T *value, const char *function_name)` +returning `non_det_log_t *`, with `T` the C type develop's +`NonDetGenerator.h` gives the family (`bool` is `int`, `loff_t` is +`long long`, `sector_t` is `unsigned long`; `pointer` and `pchar` pass a +`void **` and a `char **`), the decision of #13. The pass stores the call's result in an `alloca` of that type and +passes its address; for `bool` it widens the `_Bool` to `int` first. The +`klee_log.csv` row is the one develop writes: the `enum NONDET_TYPE` +number of the export (`Map2CheckTypes.h:41-58`), and the value as +`NonDetPass.cpp` would have passed it to develop's `int` wrapper (narrow +types zero-extended, wide types and addresses truncated to 32 bits), +printed with `%u` for `unsigned`, `%lf` for `double` and `%d` otherwise +(`NonDetLog.c:45-54`). The golden `tests/legacy_golden/NONDET-TYPES/` holds +one row of each type. `type_var` is only kept in the returned record. + ## Discontinued: `map2check_gen_data_fuzzer_pchar` / `_loff_t` / `_sector_t` **Status:** discontinued, not ported to `feature/rust`. diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 981aad9..d736580 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -517,7 +517,7 @@ map2check_map_malloc(NULL, 4); /* writes map2check_property__unknown, aborts * ### SD-17 — The memory and nondet logs are rebuilt from this library's records **Classification:** accepted limitation -**Tests:** `rust/src/ffi_golden_tests.rs::false_free_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::false_free_resolved_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::none_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::false_memtrack_matches_c_runtime`, `rust/src/ffi_tests.rs::list_log_free_row_names_the_variable_passed_to_free`, `rust/src/ffi_tests.rs::failing_free_keeps_its_list_log_row_like_develop`, `rust/src/output/legacy_files.rs::nondet_rows_use_develop_types_and_value_formats` +**Tests:** `rust/src/ffi_golden_tests.rs::false_free_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::false_free_resolved_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::none_matches_c_runtime`, `rust/src/ffi_golden_tests.rs::false_memtrack_matches_c_runtime`, `rust/src/ffi_tests.rs::list_log_free_row_names_the_variable_passed_to_free`, `rust/src/ffi_tests.rs::failing_free_keeps_its_list_log_row_like_develop`, `rust/src/output/legacy_files.rs::nondet_rows_use_develop_types_and_value_formats`, `rust/src/ffi_golden_tests.rs::nondet_types_match_c_runtime` **Rust:** `heap_log.csv`, `allocation_log.csv`, `list_log.csv` and `klee_log.csv` are rebuilt at the end of the run by replaying the memory @@ -533,7 +533,11 @@ appends before it checks (`AnalysisModeMemtrack.c:346-358`); the golden - **`klee_log.csv`.** Written on every violating witness run. Develop writes it only with the AFL++ or KLEE generator; `NonDetGeneratorNone.c` writes none (`:42`). A `float` value, which develop has no logger for, is - logged as a double (type 15). + logged as a double (type 15). Every other type matches develop byte for + byte, value format and type number included: the value is the one + `NonDetPass.cpp` hands to develop's `int` wrapper (narrow types + zero-extended, so a `char` -3 is `253`; wide types and addresses + truncated to 32 bits), as the golden `NONDET-TYPES` checks. - **Long function names.** Develop copies names into `char[35]` with `strncpy` (`FUNCTION_MAX_LENGTH_NAME`), which leaves names of 35 or more characters unterminated; this library writes the whole name. diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index 21d0838..ea26e1e 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -411,6 +411,8 @@ pub union LegacyNonDetData { pub str: [std::os::raw::c_char; 20], pub f: *mut f32, pub d: *mut f64, + /// The `value` pointer of the families with no typed member above. + pub p: *mut std::os::raw::c_void, } impl std::fmt::Debug for LegacyNonDetData { @@ -424,10 +426,13 @@ impl std::fmt::Debug for LegacyNonDetData { unsafe impl Send for non_det_log_t {} macro_rules! nondet_ffi { - ($fn_name:ident, $variant:ident, $rust_ty:ty, $c_ty:ty, $field:ident) => { + ($fn_name:ident, $c_ty:ty, $field:ident, $zero:expr, $to_value:expr) => { /// # Safety /// `value` must be null or point to a valid value of the expected type. /// `function_name` must be null or a valid null-terminated C string. + // `c_long` and friends are `i64`/`u64` on LP64 only: the widening + // conversions in `$to_value` are no-ops there and real elsewhere. + #[allow(clippy::useless_conversion)] #[no_mangle] pub unsafe extern "C" fn $fn_name( line: i32, @@ -440,12 +445,13 @@ macro_rules! nondet_ffi { || { // SAFETY: caller guarantees function_name is a valid C string or null. let fname = unsafe { c_str(function_name) }; - let raw_value = if value.is_null() { - <$c_ty>::default() + let raw_value: $c_ty = if value.is_null() { + $zero } else { // SAFETY: caller guarantees value points to a valid value. unsafe { *value } }; + let to_value: fn($c_ty) -> NonDetValue = $to_value; match state::with_state(|s| { let step = s.next_step(); s.nondets.push(NonDetEntry::new( @@ -453,7 +459,7 @@ macro_rules! nondet_ffi { line as u32, scope as u32, fname, - NonDetValue::$variant(raw_value as $rust_ty), + to_value(raw_value), )); let entry = Box::new(non_det_log_t { tqe_next: std::ptr::null_mut(), @@ -462,7 +468,7 @@ macro_rules! nondet_ffi { line, scope, type_var, - value: LegacyNonDetData { $field: value }, + value: LegacyNonDetData { $field: value as _ }, function_name, }); let pointer = (&*entry) as *const non_det_log_t as *mut non_det_log_t; @@ -482,24 +488,114 @@ macro_rules! nondet_ffi { }; } -nondet_ffi!(map2check_save_nondet_log_int, Int, i32, i32, i); -nondet_ffi!(map2check_save_nondet_log_uint, UInt, u32, u32, u); +// One export per `map2check_nondet_` of develop (`NonDetLog.c:96-130`), +// each taking a pointer to the value in the C type `NonDetGenerator.h` gives +// that family. The `type_var` argument is only kept in the returned record: +// the `enum NONDET_TYPE` of the row comes from the export, as in develop. +nondet_ffi!(map2check_save_nondet_log_int, i32, i, 0, NonDetValue::Int); +nondet_ffi!(map2check_save_nondet_log_uint, u32, u, 0, NonDetValue::UInt); +nondet_ffi!( + map2check_save_nondet_log_unsigned, + std::os::raw::c_uint, + u, + 0, + NonDetValue::Unsigned +); nondet_ffi!( map2check_save_nondet_log_long, - Long, - i64, std::os::raw::c_long, - l + l, + 0, + |v| NonDetValue::Long(i64::from(v)) +); +nondet_ffi!( + map2check_save_nondet_log_ulong, + std::os::raw::c_ulong, + p, + 0, + |v| NonDetValue::ULong(u64::from(v)) ); nondet_ffi!( map2check_save_nondet_log_char, - Char, - i8, std::os::raw::c_char, - c + c, + 0, + |v| NonDetValue::Char(i8::from_ne_bytes([v as u8])) +); +nondet_ffi!( + map2check_save_nondet_log_uchar, + u8, + p, + 0, + NonDetValue::UChar +); +nondet_ffi!( + map2check_save_nondet_log_short, + i16, + p, + 0, + NonDetValue::Short +); +nondet_ffi!( + map2check_save_nondet_log_ushort, + u16, + p, + 0, + NonDetValue::UShort +); +// develop: `typedef int bool` (NonDetGenerator.h:74). +nondet_ffi!(map2check_save_nondet_log_bool, i32, p, 0, NonDetValue::Bool); +nondet_ffi!( + map2check_save_nondet_log_size_t, + usize, + p, + 0, + NonDetValue::SizeT +); +// glibc: `typedef long long loff_t`. +nondet_ffi!( + map2check_save_nondet_log_loff_t, + std::os::raw::c_longlong, + p, + 0, + |v| NonDetValue::LoffT(i64::from(v)) +); +// develop: `#define sector_t unsigned long` (NonDetGenerator.h:112). +nondet_ffi!( + map2check_save_nondet_log_sector_t, + std::os::raw::c_ulong, + p, + 0, + |v| NonDetValue::SectorT(u64::from(v)) +); +nondet_ffi!( + map2check_save_nondet_log_pointer, + *mut std::os::raw::c_void, + p, + std::ptr::null_mut(), + |v| NonDetValue::Pointer(v as usize) +); +nondet_ffi!( + map2check_save_nondet_log_pchar, + *mut std::os::raw::c_char, + p, + std::ptr::null_mut(), + |v| NonDetValue::PChar(v as usize) +); +nondet_ffi!( + map2check_save_nondet_log_float, + f32, + f, + 0.0, + NonDetValue::Float +); +nondet_ffi!( + map2check_save_nondet_log_double, + f64, + d, + 0.0, + NonDetValue::Double ); -nondet_ffi!(map2check_save_nondet_log_float, Float, f32, f32, f); -nondet_ffi!(map2check_save_nondet_log_double, Double, f64, f64, d); // ── Basic-block tracking ────────────────────────────────────────────────────── diff --git a/rust/src/ffi_golden_tests.rs b/rust/src/ffi_golden_tests.rs index 34f4d21..344b015 100644 --- a/rust/src/ffi_golden_tests.rs +++ b/rust/src/ffi_golden_tests.rs @@ -433,6 +433,60 @@ fn target_reached_matches_c_runtime() { assert_matches_golden("TARGET-REACHED", "target_reached_child"); } +/// `drivers/NONDET-TYPES.c`: one nondet value of every develop type, each +/// handed to its `map2check_save_nondet_log_` in its own C type, then a +/// failed assert, so the witness writes `klee_log.csv`. +#[test] +fn nondet_types_child() { + if is_child() { + start(AnalysisMode::Assert); + let f = c"f".as_ptr(); + let mut int_v: i32 = -5; + let mut unsigned_v: u32 = 4_000_000_000; + let mut char_v: c_char = -3; + let mut pointer_v: *mut c_void = 0x1_2345_6789_usize as *mut c_void; + let mut long_v: c_long = -7; + let mut ushort_v: u16 = 65535; + let mut short_v: i16 = -2; + let mut ulong_v: std::os::raw::c_ulong = 0xFFFF_FFFF_8000_0000; + let mut bool_v: i32 = 1; + let mut uchar_v: u8 = 200; + let mut pchar_v: *mut c_char = 0x7fff_1234_5678_usize as *mut c_char; + let mut size_t_v: usize = 5_000_000_000; + let mut loff_t_v: i64 = -1; + let mut sector_t_v: std::os::raw::c_ulong = 0x1_0000_0009; + let mut uint_v: u32 = 4_000_000_000; + let mut double_v: f64 = -0.125; + // SAFETY: every value outlives its call; the pointers are only + // recorded, never dereferenced; `f` is a C-string literal. + unsafe { + map2check_save_nondet_log_int(30, 1, 0, &mut int_v, f); + map2check_save_nondet_log_unsigned(31, 1, 5, &mut unsigned_v, f); + map2check_save_nondet_log_char(32, 1, 1, &mut char_v, f); + map2check_save_nondet_log_pointer(33, 1, 2, &mut pointer_v, f); + map2check_save_nondet_log_long(34, 1, 4, &mut long_v, f); + map2check_save_nondet_log_ushort(35, 1, 3, &mut ushort_v, f); + map2check_save_nondet_log_short(36, 1, 14, &mut short_v, f); + map2check_save_nondet_log_ulong(37, 1, 6, &mut ulong_v, f); + map2check_save_nondet_log_bool(38, 1, 7, &mut bool_v, f); + map2check_save_nondet_log_uchar(39, 1, 8, &mut uchar_v, f); + map2check_save_nondet_log_pchar(40, 1, 9, &mut pchar_v, f); + map2check_save_nondet_log_size_t(41, 1, 10, &mut size_t_v, f); + map2check_save_nondet_log_loff_t(42, 1, 11, &mut loff_t_v, f); + map2check_save_nondet_log_sector_t(43, 1, 12, &mut sector_t_v, f); + map2check_save_nondet_log_uint(44, 1, 13, &mut uint_v, f); + map2check_save_nondet_log_double(45, 1, 15, &mut double_v, f); + map2check_is_valid_assert(46, f, 0); + } + map2check_success(); + } +} + +#[test] +fn nondet_types_match_c_runtime() { + assert_matches_golden("NONDET-TYPES", "nondet_types_child"); +} + // ── Without witness: property files only (WitnessGenerationNone.c) ────────── #[test] diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 2039cb2..482ae12 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -300,6 +300,109 @@ fn save_nondet_log_double_records_value() { assert_nondet(record, NonDetValue::Double(-2.25)); } +/// One test per family added for the Map2Check passes (#6): the value is +/// read through the pointer in the family's C type, and the record keeps +/// the `type_var` the pass passed. +macro_rules! save_nondet_test { + ($test:ident, $c_ty:ty, $value:expr, $expected:expr, |$p:ident| $call:expr) => { + #[test] + fn $test() { + let _guard = fresh(); + let mut value: $c_ty = $value; + let $p: *mut $c_ty = &mut value; + // SAFETY: value and the C-string literal outlive the call. + let record = unsafe { $call }; + assert_nondet(record, $expected); + // SAFETY: the record is owned by the global state (see above). + assert_eq!(unsafe { (*record).type_var }, 9); + // A null value is accepted and records the family's zero. + state::reset(); + let $p: *mut $c_ty = std::ptr::null_mut(); + // SAFETY: as above. + let record = unsafe { $call }; + assert!(!record.is_null()); + } + }; +} + +save_nondet_test!( + save_nondet_log_unsigned_records_value, + std::os::raw::c_uint, + 4_000_000_000, + NonDetValue::Unsigned(4_000_000_000), + |p| map2check_save_nondet_log_unsigned(10, 2, 9, p, c"f".as_ptr()) +); +save_nondet_test!( + save_nondet_log_ulong_records_value, + std::os::raw::c_ulong, + u64::MAX as std::os::raw::c_ulong, + NonDetValue::ULong(u64::MAX), + |p| map2check_save_nondet_log_ulong(10, 2, 9, p, c"f".as_ptr()) +); +save_nondet_test!( + save_nondet_log_uchar_records_value, + u8, + 200, + NonDetValue::UChar(200), + |p| map2check_save_nondet_log_uchar(10, 2, 9, p, c"f".as_ptr()) +); +save_nondet_test!( + save_nondet_log_short_records_value, + i16, + -2, + NonDetValue::Short(-2), + |p| map2check_save_nondet_log_short(10, 2, 9, p, c"f".as_ptr()) +); +save_nondet_test!( + save_nondet_log_ushort_records_value, + u16, + 65535, + NonDetValue::UShort(65535), + |p| map2check_save_nondet_log_ushort(10, 2, 9, p, c"f".as_ptr()) +); +save_nondet_test!( + save_nondet_log_bool_records_value, + i32, + 1, + NonDetValue::Bool(1), + |p| map2check_save_nondet_log_bool(10, 2, 9, p, c"f".as_ptr()) +); +save_nondet_test!( + save_nondet_log_size_t_records_value, + usize, + 5_000_000_000, + NonDetValue::SizeT(5_000_000_000), + |p| map2check_save_nondet_log_size_t(10, 2, 9, p, c"f".as_ptr()) +); +save_nondet_test!( + save_nondet_log_loff_t_records_value, + std::os::raw::c_longlong, + -1, + NonDetValue::LoffT(-1), + |p| map2check_save_nondet_log_loff_t(10, 2, 9, p, c"f".as_ptr()) +); +save_nondet_test!( + save_nondet_log_sector_t_records_value, + std::os::raw::c_ulong, + 0x1_0000_0009, + NonDetValue::SectorT(0x1_0000_0009), + |p| map2check_save_nondet_log_sector_t(10, 2, 9, p, c"f".as_ptr()) +); +save_nondet_test!( + save_nondet_log_pointer_records_address, + *mut c_void, + 0x1_2345_6789_usize as *mut c_void, + NonDetValue::Pointer(0x1_2345_6789), + |p| map2check_save_nondet_log_pointer(10, 2, 9, p, c"f".as_ptr()) +); +save_nondet_test!( + save_nondet_log_pchar_records_address, + *mut std::os::raw::c_char, + 0x7fff_1234_5678_usize as *mut std::os::raw::c_char, + NonDetValue::PChar(0x7fff_1234_5678), + |p| map2check_save_nondet_log_pchar(10, 2, 9, p, c"f".as_ptr()) +); + // ── Basic-block tracking ──────────────────────────────────────────────────── #[test] @@ -1648,6 +1751,18 @@ fn macro_generated_exports_match_the_hand_written_header_declarations() { let _: NonDet = map2check_save_nondet_log_char; // char *value let _: NonDet = map2check_save_nondet_log_float; // float *value let _: NonDet = map2check_save_nondet_log_double; // double *value + use std::os::raw::{c_longlong, c_uint, c_ulong}; + let _: NonDet = map2check_save_nondet_log_unsigned; // unsigned int *value + let _: NonDet = map2check_save_nondet_log_ulong; // unsigned long *value + let _: NonDet = map2check_save_nondet_log_uchar; // unsigned char *value + let _: NonDet = map2check_save_nondet_log_short; // short *value + let _: NonDet = map2check_save_nondet_log_ushort; // unsigned short *value + let _: NonDet = map2check_save_nondet_log_bool; // int *value + let _: NonDet = map2check_save_nondet_log_size_t; // size_t *value + let _: NonDet = map2check_save_nondet_log_loff_t; // long long *value + let _: NonDet = map2check_save_nondet_log_sector_t; // unsigned long *value + let _: NonDet<*mut c_void> = map2check_save_nondet_log_pointer; // void **value + let _: NonDet<*mut c_char> = map2check_save_nondet_log_pchar; // char **value let _: BinOp = map2check_binop_add_int; let _: BinOp = map2check_binop_sub_int; let _: BinOp = map2check_binop_mul_int; diff --git a/rust/src/nondet/mod.rs b/rust/src/nondet/mod.rs index b7e7ffe..c21bfee 100644 --- a/rust/src/nondet/mod.rs +++ b/rust/src/nondet/mod.rs @@ -5,6 +5,12 @@ use serde::Serialize; /// Typed nondeterministic value — replaces `union Data` + `enum var_type_t`. /// Each variant carries the actual value, eliminating unsafe type-punning. +/// +/// One variant per `enum NONDET_TYPE` of Map2Check `develop` +/// (`Map2CheckTypes.h:41-58`), with the C type `NonDetGenerator.h` gives it +/// (`bool` is `typedef int bool`, `sector_t` is `unsigned long`, `loff_t` is +/// glibc's `long long`, `pointer` and `pchar` are addresses), plus `Float`, +/// which develop does not instrument. #[derive(Debug, Clone, Serialize, PartialEq)] #[serde(tag = "type", content = "value")] pub enum NonDetValue { @@ -20,6 +26,28 @@ pub enum NonDetValue { Float(f32), #[serde(rename = "DOUBLE")] Double(f64), + #[serde(rename = "UNSIGNED")] + Unsigned(u32), + #[serde(rename = "POINTER")] + Pointer(usize), + #[serde(rename = "USHORT")] + UShort(u16), + #[serde(rename = "SHORT")] + Short(i16), + #[serde(rename = "ULONG")] + ULong(u64), + #[serde(rename = "BOOL")] + Bool(i32), + #[serde(rename = "UCHAR")] + UChar(u8), + #[serde(rename = "PCHAR")] + PChar(usize), + #[serde(rename = "SIZE_T")] + SizeT(usize), + #[serde(rename = "LOFF_T")] + LoffT(i64), + #[serde(rename = "SECTOR_T")] + SectorT(u64), } impl NonDetValue { @@ -31,6 +59,17 @@ impl NonDetValue { Self::Char(_) => "CHAR", Self::Float(_) => "FLOAT", Self::Double(_) => "DOUBLE", + Self::Unsigned(_) => "UNSIGNED", + Self::Pointer(_) => "POINTER", + Self::UShort(_) => "USHORT", + Self::Short(_) => "SHORT", + Self::ULong(_) => "ULONG", + Self::Bool(_) => "BOOL", + Self::UChar(_) => "UCHAR", + Self::PChar(_) => "PCHAR", + Self::SizeT(_) => "SIZE_T", + Self::LoffT(_) => "LOFF_T", + Self::SectorT(_) => "SECTOR_T", } } } @@ -103,6 +142,17 @@ mod tests { (NonDetValue::Char(-1), "CHAR"), (NonDetValue::Float(0.5), "FLOAT"), (NonDetValue::Double(0.5), "DOUBLE"), + (NonDetValue::Unsigned(1), "UNSIGNED"), + (NonDetValue::Pointer(1), "POINTER"), + (NonDetValue::UShort(1), "USHORT"), + (NonDetValue::Short(-1), "SHORT"), + (NonDetValue::ULong(1), "ULONG"), + (NonDetValue::Bool(1), "BOOL"), + (NonDetValue::UChar(1), "UCHAR"), + (NonDetValue::PChar(1), "PCHAR"), + (NonDetValue::SizeT(1), "SIZE_T"), + (NonDetValue::LoffT(-1), "LOFF_T"), + (NonDetValue::SectorT(1), "SECTOR_T"), ]; for (value, name) in cases { assert_eq!(value.type_name(), name); diff --git a/rust/src/output/legacy_files.rs b/rust/src/output/legacy_files.rs index 3f83745..43467ba 100644 --- a/rust/src/output/legacy_files.rs +++ b/rust/src/output/legacy_files.rs @@ -170,17 +170,40 @@ pub fn write_trackbb_log(dir: &Path, rows: &[BasicBlockEntry]) -> io::Result<()> } /// The value column and the `enum NONDET_TYPE` number -/// (`Map2CheckTypes.h:41-58`) of a nondet row, as `nondet_log_to_file` -/// prints them (`NonDetLog.c:45-54`). The `map2check_nondet_*` wrappers take -/// the value as a C `int` (`NonDetLog.c:105-113`), so every integer kind is -/// printed as its low 32 bits with `%d`; a double is printed with `%lf`. -/// `float` has no develop counterpart and is logged as a double. +/// (`Map2CheckTypes.h:41-58`) of a nondet row, as develop writes them. +/// +/// `NonDetPass.cpp` hands each value to its `map2check_nondet_` wrapper as +/// a C `int` (`NonDetLog.c:96-130`): `int`, `unsigned`, `uint` as they are +/// (`NONDET_IMPL_HELPER`, `:260-280`); the other integer kinds through +/// `CreateIntegerCast(v, i32, isSigned = false)` (`:240-258`, `:281-290`), +/// which zero-extends the narrow ones (`char` -3 arrives as 253) and +/// truncates the wide ones to their low 32 bits; `pointer` and `pchar` +/// through `CreatePtrToInt(v, i32)` (`:224-238`, `:291-292`), the low 32 +/// bits of the address. `nondet_log_to_file` then prints `UNSIGNED` with +/// `%u`, `DOUBLE` with `%lf` and every other type with `%d` +/// (`NonDetLog.c:45-54`). `float` has no develop counterpart and is logged as +/// a double. pub fn nondet_value_and_type(value: &NonDetValue) -> (String, i32) { + /// `trunc` to `i32`, as `CreateIntegerCast` narrows a wide integer. + fn low32(v: u64) -> String { + (v as u32 as i32).to_string() + } match *value { NonDetValue::Int(v) => (v.to_string(), 0), - NonDetValue::Char(v) => ((v as i32).to_string(), 1), - NonDetValue::Long(v) => ((v as i32).to_string(), 4), + NonDetValue::Char(v) => ((v as u8).to_string(), 1), + NonDetValue::Pointer(v) => (low32(v as u64), 2), + NonDetValue::UShort(v) => (v.to_string(), 3), + NonDetValue::Long(v) => (low32(v as u64), 4), + NonDetValue::Unsigned(v) => (v.to_string(), 5), + NonDetValue::ULong(v) => (low32(v), 6), + NonDetValue::Bool(v) => (v.to_string(), 7), + NonDetValue::UChar(v) => (v.to_string(), 8), + NonDetValue::PChar(v) => (low32(v as u64), 9), + NonDetValue::SizeT(v) => (low32(v as u64), 10), + NonDetValue::LoffT(v) => (low32(v as u64), 11), + NonDetValue::SectorT(v) => (low32(v), 12), NonDetValue::UInt(v) => ((v as i32).to_string(), 13), + NonDetValue::Short(v) => ((v as u16).to_string(), 14), NonDetValue::Float(v) => (c_double(f64::from(v)), 15), NonDetValue::Double(v) => (c_double(v), 15), } @@ -768,7 +791,16 @@ mod tests { assert_eq!(nondet_value_and_type(&V::Int(-1)), ("-1".into(), 0)); assert_eq!(nondet_value_and_type(&V::UInt(u32::MAX)), ("-1".into(), 13)); assert_eq!(nondet_value_and_type(&V::Long(1 << 32)), ("0".into(), 4)); - assert_eq!(nondet_value_and_type(&V::Char(-3)), ("-3".into(), 1)); + assert_eq!(nondet_value_and_type(&V::Char(-3)), ("253".into(), 1)); + assert_eq!(nondet_value_and_type(&V::Short(-2)), ("65534".into(), 14)); + assert_eq!( + nondet_value_and_type(&V::Unsigned(u32::MAX)), + ("4294967295".into(), 5) + ); + assert_eq!( + nondet_value_and_type(&V::Pointer(0x1_2345_6789)), + ("591751049".into(), 2) + ); assert_eq!( nondet_value_and_type(&V::Float(0.5)), ("0.500000".into(), 15) diff --git a/tests/legacy_golden/ASSERT/klee_log.csv b/tests/legacy_golden/ASSERT/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/ASSERT/klee_log.csv +++ b/tests/legacy_golden/ASSERT/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/DIVBYZERO/klee_log.csv b/tests/legacy_golden/DIVBYZERO/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/DIVBYZERO/klee_log.csv +++ b/tests/legacy_golden/DIVBYZERO/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/FALSE-DEREF/klee_log.csv b/tests/legacy_golden/FALSE-DEREF/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/FALSE-DEREF/klee_log.csv +++ b/tests/legacy_golden/FALSE-DEREF/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/FALSE-FREE-RESOLVED/klee_log.csv b/tests/legacy_golden/FALSE-FREE-RESOLVED/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/FALSE-FREE-RESOLVED/klee_log.csv +++ b/tests/legacy_golden/FALSE-FREE-RESOLVED/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/FALSE-FREE/klee_log.csv b/tests/legacy_golden/FALSE-FREE/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/FALSE-FREE/klee_log.csv +++ b/tests/legacy_golden/FALSE-FREE/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/FALSE-MEMCLEANUP/klee_log.csv b/tests/legacy_golden/FALSE-MEMCLEANUP/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/FALSE-MEMCLEANUP/klee_log.csv +++ b/tests/legacy_golden/FALSE-MEMCLEANUP/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/FALSE-MEMTRACK/klee_log.csv b/tests/legacy_golden/FALSE-MEMTRACK/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/FALSE-MEMTRACK/klee_log.csv +++ b/tests/legacy_golden/FALSE-MEMTRACK/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/NONDET-TYPES/.exit_status b/tests/legacy_golden/NONDET-TYPES/.exit_status new file mode 100644 index 0000000..405e2af --- /dev/null +++ b/tests/legacy_golden/NONDET-TYPES/.exit_status @@ -0,0 +1 @@ +134 diff --git a/tests/legacy_golden/NONDET-TYPES/klee_log.csv b/tests/legacy_golden/NONDET-TYPES/klee_log.csv new file mode 100644 index 0000000..a87e69c --- /dev/null +++ b/tests/legacy_golden/NONDET-TYPES/klee_log.csv @@ -0,0 +1,21 @@ +0;5;0;main;0;42;0 +1;6;0;main;1;-294967296;13 +2;7;0;main;2;253;1 +3;8;0;main;3;-7;4 +4;9;0;main;4;2.500000;15 +5;30;1;f;5;-5;0 +6;31;1;f;6;4000000000;5 +7;32;1;f;7;253;1 +8;33;1;f;8;591751049;2 +9;34;1;f;9;-7;4 +10;35;1;f;10;65535;3 +11;36;1;f;11;65534;14 +12;37;1;f;12;-2147483648;6 +13;38;1;f;13;1;7 +14;39;1;f;14;200;8 +15;40;1;f;15;305419896;9 +16;41;1;f;16;705032704;10 +17;42;1;f;17;-1;11 +18;43;1;f;18;9;12 +19;44;1;f;19;-294967296;13 +20;45;1;f;20;-0.125000;15 diff --git a/tests/legacy_golden/NONDET-TYPES/map2check_checked_error b/tests/legacy_golden/NONDET-TYPES/map2check_checked_error new file mode 100644 index 0000000..464ee54 --- /dev/null +++ b/tests/legacy_golden/NONDET-TYPES/map2check_checked_error @@ -0,0 +1 @@ +END OF TIME diff --git a/tests/legacy_golden/NONDET-TYPES/map2check_property b/tests/legacy_golden/NONDET-TYPES/map2check_property new file mode 100644 index 0000000..820b5ce --- /dev/null +++ b/tests/legacy_golden/NONDET-TYPES/map2check_property @@ -0,0 +1,3 @@ +ASSERT +Line: 46 +Function: f diff --git a/tests/legacy_golden/NONDET-TYPES/map2check_property_assert b/tests/legacy_golden/NONDET-TYPES/map2check_property_assert new file mode 100644 index 0000000..820b5ce --- /dev/null +++ b/tests/legacy_golden/NONDET-TYPES/map2check_property_assert @@ -0,0 +1,3 @@ +ASSERT +Line: 46 +Function: f diff --git a/tests/legacy_golden/NONDET-TYPES/track_bb_log.st b/tests/legacy_golden/NONDET-TYPES/track_bb_log.st new file mode 100644 index 0000000..03bf5b6 --- /dev/null +++ b/tests/legacy_golden/NONDET-TYPES/track_bb_log.st @@ -0,0 +1,2 @@ +3;main +4;main diff --git a/tests/legacy_golden/OVERFLOW/klee_log.csv b/tests/legacy_golden/OVERFLOW/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/OVERFLOW/klee_log.csv +++ b/tests/legacy_golden/OVERFLOW/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/POSIX-MEMALIGN/klee_log.csv b/tests/legacy_golden/POSIX-MEMALIGN/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/POSIX-MEMALIGN/klee_log.csv +++ b/tests/legacy_golden/POSIX-MEMALIGN/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/TARGET-REACHED/klee_log.csv b/tests/legacy_golden/TARGET-REACHED/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/TARGET-REACHED/klee_log.csv +++ b/tests/legacy_golden/TARGET-REACHED/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/UNKNOWN/klee_log.csv b/tests/legacy_golden/UNKNOWN/klee_log.csv index dadd065..662c5f2 100644 --- a/tests/legacy_golden/UNKNOWN/klee_log.csv +++ b/tests/legacy_golden/UNKNOWN/klee_log.csv @@ -1,5 +1,5 @@ 0;5;0;main;0;42;0 1;6;0;main;1;-294967296;13 -2;7;0;main;2;-3;1 +2;7;0;main;2;253;1 3;8;0;main;3;-7;4 4;9;0;main;4;2.500000;15 diff --git a/tests/legacy_golden/drivers/NONDET-TYPES.c b/tests/legacy_golden/drivers/NONDET-TYPES.c new file mode 100644 index 0000000..0040843 --- /dev/null +++ b/tests/legacy_golden/drivers/NONDET-TYPES.c @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Assert mode: one nondet value of every NONDET_TYPE (Map2CheckTypes.h:41-58), + * then a failed assert, so the witness writes klee_log.csv. + * + * Each value reaches its map2check_nondet_ wrapper (NonDetLog.c:96-130) + * as NonDetPass.cpp passes it: + * - int, unsigned, uint, double: the call result itself (NONDET_IMPL_HELPER, + * :260-275, used at :277-280); + * - char, ushort, short, long, ulong, bool, uchar, size_t, loff_t, + * sector_t: CreateIntegerCast(v, i32, isSigned = false) (:240-258, + * :281-290), a zero extension of the narrow types and a truncation of the + * wide ones; + * - pchar, pointer: CreatePtrToInt(v, i32) (:224-238, :291-292), the low 32 + * bits of the address. + * bool is the `_Bool` of __VERIFIER_nondet_bool, an i1, so it reaches the + * wrapper as 0 or 1. + */ +#include "golden.h" + +void map2check_nondet_unsigned(unsigned line, unsigned scope, unsigned value, + const char *function_name); +void map2check_nondet_pointer(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_ushort(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_short(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_ulong(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_bool(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_uchar(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_pchar(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_size_t(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_loff_t(unsigned line, unsigned scope, int value, + const char *function_name); +void map2check_nondet_sector_t(unsigned line, unsigned scope, int value, + const char *function_name); + +/* The LLVM casts, in C: zext of a narrow value, trunc of a wide one. */ +#define ZEXT8(v) ((int)(unsigned char)(v)) +#define ZEXT16(v) ((int)(unsigned short)(v)) +#define TRUNC(v) ((int)(unsigned)(unsigned long long)(v)) + +int __map2check_main__(void) { + map2check_init(0); + golden_tracking(); + map2check_nondet_int(30, 1, -5, "f"); + map2check_nondet_unsigned(31, 1, 4000000000u, "f"); + map2check_nondet_char(32, 1, ZEXT8((char)-3), "f"); + map2check_nondet_pointer(33, 1, TRUNC(0x123456789ULL), "f"); + map2check_nondet_long(34, 1, TRUNC(-7L), "f"); + map2check_nondet_ushort(35, 1, ZEXT16((unsigned short)65535), "f"); + map2check_nondet_short(36, 1, ZEXT16((short)-2), "f"); + map2check_nondet_ulong(37, 1, TRUNC(0xFFFFFFFF80000000UL), "f"); + map2check_nondet_bool(38, 1, 1, "f"); + map2check_nondet_uchar(39, 1, ZEXT8((unsigned char)200), "f"); + map2check_nondet_pchar(40, 1, TRUNC(0x7fff12345678ULL), "f"); + map2check_nondet_size_t(41, 1, TRUNC(5000000000UL), "f"); + map2check_nondet_loff_t(42, 1, TRUNC(-1LL), "f"); + map2check_nondet_sector_t(43, 1, TRUNC(0x100000009UL), "f"); + map2check_nondet_uint(44, 1, (int)4000000000u, "f"); + map2check_nondet_double(45, 1, -0.125, "f"); + map2check_assert(0, 46, "f"); + map2check_success(); + return 0; +} diff --git a/tests/legacy_golden/drivers/build_goldens.sh b/tests/legacy_golden/drivers/build_goldens.sh index 4bea45b..db38e7b 100755 --- a/tests/legacy_golden/drivers/build_goldens.sh +++ b/tests/legacy_golden/drivers/build_goldens.sh @@ -48,10 +48,12 @@ declare -A MODE=( [POSIX-MEMALIGN]="AnalysisModeMemtrack $MEMORY" [ASSERT]="AnalysisModeAssert" [TARGET-REACHED]="AnalysisModeNone" + [NONDET-TYPES]="AnalysisModeAssert" ) for tag in FALSE-FREE FALSE-FREE-RESOLVED FALSE-DEREF FALSE-MEMTRACK FALSE-MEMCLEANUP NONE \ - UNKNOWN OVERFLOW DIVBYZERO POSIX-MEMALIGN ASSERT TARGET-REACHED; do + UNKNOWN OVERFLOW DIVBYZERO POSIX-MEMALIGN ASSERT TARGET-REACHED \ + NONDET-TYPES; do bc "$DRIVERS/$tag.c" "$WORK/$tag-output.bc" inputs=("$WORK/$tag-output.bc") for f in $COMMON ${MODE[$tag]} NonDetGeneratorLog WitnessGeneration; do diff --git a/tests/legacy_golden/drivers/golden.h b/tests/legacy_golden/drivers/golden.h index 42612db..e0f5987 100644 --- a/tests/legacy_golden/drivers/golden.h +++ b/tests/legacy_golden/drivers/golden.h @@ -94,7 +94,9 @@ static void golden_tracking(void) { map2check_track_bb(3, "main"); map2check_nondet_int(5, 0, 42, "main"); map2check_nondet_uint(6, 0, (int)4000000000u, "main"); - map2check_nondet_char(7, 0, -3, "main"); + /* NonDetPass.cpp:240-258,281 widens a char with CreateIntegerCast(..., false), + * a zero extension: the wrapper receives (unsigned char)-3. */ + map2check_nondet_char(7, 0, (int)(unsigned char)(char)-3, "main"); map2check_nondet_long(8, 0, -7, "main"); map2check_nondet_double(9, 0, 2.5, "main"); } From f4e4908302afa687a48acb21e552d5ee40b62197 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 23:20:14 -0400 Subject: [PATCH 31/63] feat: read fuzzer input like develop's AFL++ generator and reject inputs on a failed assume (#6, #13) The __VERIFIER_nondet_* generators now consume the input exactly as NonDetGeneratorAFL.c does (D1 conditions 4 and 9): sizeof(type) bytes per value, in memory order, and past the end of the input the reader rewinds, returning data[0] once without advancing (get_next_input_from_afl). A short input no longer reads as 0: the seed "A" gives 0x41414141, which CmpLog can match. bool reads four bytes (develop's typedef int bool). Add __VERIFIER_nondet_{pchar,loff_t,sector_t}: pchar reads an unsigned length, NULL for 0, capped at 4096, then length - 1 chars from the C malloc, as develop. __VERIFIER_assume(0) and map2check_fuzzer_assume(0) inside LLVMFuzzerTestOneInput now abandon the current input and the fuzzer continues, as develop's nondet_cancel longjmp does, instead of exit(0). They resume_unwind a private payload through C-unwind frames and LLVMFuzzerTestOneInput catches it; a longjmp would skip a Rust frame with a destructor. Outside an iteration, or with panic = "abort", they still exit(0). The C harness and codefuzz test cross the unwind through a C frame and fail if the process ends early. --- docs/api-deltas.md | 153 +++++---- docs/semantic-deltas.md | 88 ++++- rust/src/nondet/libfuzzer.rs | 594 +++++++++++++++++++++++++-------- test/codefuzz_by_caller_test.c | 45 ++- tests/ffi_harness.c | 114 +++++++ tests/legacy_golden/README.md | 3 +- 6 files changed, 772 insertions(+), 225 deletions(-) diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 5f14c9d..5057498 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -63,17 +63,17 @@ build with `--features libfuzzer`. | `map2check_save_nondet_log_float` | `non_det_log_t *map2check_save_nondet_log_float(int line, int scope, enum var_type_t type_var, float *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_double` | `non_det_log_t *map2check_save_nondet_log_double(int line, int scope, enum var_type_t type_var, double *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_char` | `non_det_log_t *map2check_save_nondet_log_char(int line, int scope, enum var_type_t type_var, char *value, const char *function_name)` | returns `non_det_log_t *`; `char` value | same | `char` is `i8` on the Rust side; see “`char` ABI”. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_unsigned` | `non_det_log_t *map2check_save_nondet_log_unsigned(int line, int scope, int type_var, unsigned int *value, const char *function_name)` | — | new | Develop `map2check_nondet_unsigned` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UNSIGNED (5), value printed with `%u`; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_ulong` | `non_det_log_t *map2check_save_nondet_log_ulong(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_ulong` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type ULONG (6), low 32 bits with `%d`; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_uchar` | `non_det_log_t *map2check_save_nondet_log_uchar(int line, int scope, int type_var, unsigned char *value, const char *function_name)` | — | new | Develop `map2check_nondet_uchar` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UCHAR (8), value zero-extended; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_short` | `non_det_log_t *map2check_save_nondet_log_short(int line, int scope, int type_var, short *value, const char *function_name)` | — | new | Develop `map2check_nondet_short` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SHORT (14), value zero-extended as the pass does (`-2` is logged as `65534`); see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_ushort` | `non_det_log_t *map2check_save_nondet_log_ushort(int line, int scope, int type_var, unsigned short *value, const char *function_name)` | — | new | Develop `map2check_nondet_ushort` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type USHORT (3), value zero-extended; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_bool` | `non_det_log_t *map2check_save_nondet_log_bool(int line, int scope, int type_var, int *value, const char *function_name)` | — | new | Develop `map2check_nondet_bool` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type BOOL (7); develop's `bool` is `typedef int bool` (`NonDetGenerator.h:74`), so the pass widens the `_Bool` to `int` before storing it; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_size_t` | `non_det_log_t *map2check_save_nondet_log_size_t(int line, int scope, int type_var, size_t *value, const char *function_name)` | — | new | Develop `map2check_nondet_size_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SIZE_T (10), low 32 bits with `%d`; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_loff_t` | `non_det_log_t *map2check_save_nondet_log_loff_t(int line, int scope, int type_var, long long *value, const char *function_name)` | — | new | Develop `map2check_nondet_loff_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type LOFF_T (11); `loff_t` is glibc's `long long`; low 32 bits with `%d`; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_sector_t` | `non_det_log_t *map2check_save_nondet_log_sector_t(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_sector_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SECTOR_T (12); `sector_t` is `unsigned long` (`NonDetGenerator.h:112`); low 32 bits with `%d`; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_pointer` | `non_det_log_t *map2check_save_nondet_log_pointer(int line, int scope, int type_var, void **value, const char *function_name)` | — | new | Develop `map2check_nondet_pointer` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type POINTER (2); the address, logged as its low 32 bits (the pass's `ptrtoint` to `i32`); see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_pchar` | `non_det_log_t *map2check_save_nondet_log_pchar(int line, int scope, int type_var, char **value, const char *function_name)` | — | new | Develop `map2check_nondet_pchar` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type PCHAR (9); the string's address, logged as its low 32 bits; see “Nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_unsigned` | `non_det_log_t *map2check_save_nondet_log_unsigned(int line, int scope, int type_var, unsigned int *value, const char *function_name)` | — | new | Develop `map2check_nondet_unsigned` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UNSIGNED (5), value printed with `%u`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_ulong` | `non_det_log_t *map2check_save_nondet_log_ulong(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_ulong` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type ULONG (6), low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_uchar` | `non_det_log_t *map2check_save_nondet_log_uchar(int line, int scope, int type_var, unsigned char *value, const char *function_name)` | — | new | Develop `map2check_nondet_uchar` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UCHAR (8), value zero-extended; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_short` | `non_det_log_t *map2check_save_nondet_log_short(int line, int scope, int type_var, short *value, const char *function_name)` | — | new | Develop `map2check_nondet_short` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SHORT (14), value zero-extended as the pass does (`-2` is logged as `65534`); see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_ushort` | `non_det_log_t *map2check_save_nondet_log_ushort(int line, int scope, int type_var, unsigned short *value, const char *function_name)` | — | new | Develop `map2check_nondet_ushort` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type USHORT (3), value zero-extended; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_bool` | `non_det_log_t *map2check_save_nondet_log_bool(int line, int scope, int type_var, int *value, const char *function_name)` | — | new | Develop `map2check_nondet_bool` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type BOOL (7); develop's `bool` is `typedef int bool` (`NonDetGenerator.h:74`), so the pass widens the `_Bool` to `int` before storing it; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_size_t` | `non_det_log_t *map2check_save_nondet_log_size_t(int line, int scope, int type_var, size_t *value, const char *function_name)` | — | new | Develop `map2check_nondet_size_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SIZE_T (10), low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_loff_t` | `non_det_log_t *map2check_save_nondet_log_loff_t(int line, int scope, int type_var, long long *value, const char *function_name)` | — | new | Develop `map2check_nondet_loff_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type LOFF_T (11); `loff_t` is glibc's `long long`; low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_sector_t` | `non_det_log_t *map2check_save_nondet_log_sector_t(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_sector_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SECTOR_T (12); `sector_t` is `unsigned long` (`NonDetGenerator.h:112`); low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_pointer` | `non_det_log_t *map2check_save_nondet_log_pointer(int line, int scope, int type_var, void **value, const char *function_name)` | — | new | Develop `map2check_nondet_pointer` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type POINTER (2); the address, logged as its low 32 bits (the pass's `ptrtoint` to `i32`); see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_pchar` | `non_det_log_t *map2check_save_nondet_log_pchar(int line, int scope, int type_var, char **value, const char *function_name)` | — | new | Develop `map2check_nondet_pchar` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type PCHAR (9); the string's address, logged as its low 32 bits; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | | `map2check_save_basic_block_log` | `bbtrack_log_t *map2check_save_basic_block_log(int32_t line, const char *function_name)` | returns `bbtrack_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_is_in_trackbb_container` | `int32_t map2check_is_in_trackbb_container(int32_t line)` | `map2check_is_in_trackbb_container` | same | | | `map2check_map_alloca` | `void map2check_map_alloca(const char *var_name, const void *ptr_address, int32_t size, int32_t size_primitive, int32_t line_number, int32_t scope)` | `container_memtracklog.h:41` | same | | @@ -108,21 +108,24 @@ build with `--features libfuzzer`. | `map2check_check_deref` | `void map2check_check_deref(const void *ptr, uint32_t scope, uint32_t line, const char *function_name)` | `map2check_check_deref` | same | Name collides with the pass ABI `map2check_check_deref(i64, ptr)`; see the pass matrix (#21). Active in unset and memtrack modes only (SD-14). | | `map2check_check_mem_endprog` | `void map2check_check_mem_endprog(void)` | `map2check_check_mem_endprog` | changed | Leak algorithm rewritten. Runs `analysis_is_program_correct` of the mode: unset and memtrack report memtrack or memcleanup by option, memcleanup mode reports memcleanup only for a still-referenced last leak, the other modes nothing (SD-04, SD-14). | | `LLVMFuzzerTestOneInput` | `int32_t LLVMFuzzerTestOneInput(const uint8_t *data, uintptr_t size)` | `LLVMFuzzerTestOneInput` | same | Feature `libfuzzer`. Resets the analysis state on every iteration (SD-11). | -| `__VERIFIER_nondet_bool` | `bool __VERIFIER_nondet_bool(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_char` | `int8_t __VERIFIER_nondet_char(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_uchar` | `uint8_t __VERIFIER_nondet_uchar(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_short` | `int16_t __VERIFIER_nondet_short(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_ushort` | `uint16_t __VERIFIER_nondet_ushort(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_int` | `int32_t __VERIFIER_nondet_int(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_uint` | `uint32_t __VERIFIER_nondet_uint(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_long` | `int64_t __VERIFIER_nondet_long(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_ulong` | `uint64_t __VERIFIER_nondet_ulong(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_float` | `float __VERIFIER_nondet_float(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_double` | `double __VERIFIER_nondet_double(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_size_t` | `uintptr_t __VERIFIER_nondet_size_t(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_nondet_pointer` | `void *__VERIFIER_nondet_pointer(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. | -| `__VERIFIER_assume` | `void __VERIFIER_assume(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assume(0)` ends the process with `exit(0)` (SD-07). | -| `map2check_fuzzer_assume` | `void map2check_fuzzer_assume(int32_t condition)` | `map2check_fuzzer_assume(int)` (`nondet_gen_libfuzzer.h:41`) | same | Feature `libfuzzer`. Same `exit(0)` behaviour (SD-07). | +| `__VERIFIER_nondet_bool` | `bool __VERIFIER_nondet_bool(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads four bytes, develop's `typedef int bool`, and returns that `int != 0` (SD-18). | +| `__VERIFIER_nondet_char` | `int8_t __VERIFIER_nondet_char(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_uchar` | `uint8_t __VERIFIER_nondet_uchar(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_short` | `int16_t __VERIFIER_nondet_short(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_ushort` | `uint16_t __VERIFIER_nondet_ushort(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_int` | `int32_t __VERIFIER_nondet_int(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_uint` | `uint32_t __VERIFIER_nondet_uint(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_long` | `int64_t __VERIFIER_nondet_long(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_ulong` | `uint64_t __VERIFIER_nondet_ulong(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_float` | `float __VERIFIER_nondet_float(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads four bytes; develop has no `float` generator (SD-18). | +| `__VERIFIER_nondet_double` | `double __VERIFIER_nondet_double(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_size_t` | `uintptr_t __VERIFIER_nondet_size_t(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_pointer` | `void *__VERIFIER_nondet_pointer(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | +| `__VERIFIER_nondet_loff_t` | `int64_t __VERIFIER_nondet_loff_t(void)` | `map2check_gen_data_fuzzer_loff_t` | new | Feature `libfuzzer`. Core target of the pass's `map2check_non_det_loff_t` (#6); `loff_t` is glibc's `long long`, eight bytes (SD-18). | +| `__VERIFIER_nondet_sector_t` | `unsigned long __VERIFIER_nondet_sector_t(void)` | `map2check_gen_data_fuzzer_sector_t` | new | Feature `libfuzzer`. Core target of `map2check_non_det_sector_t` (#6); develop's `sector_t` is `unsigned long` (`NonDetGenerator.h:112`), eight bytes (SD-18). | +| `__VERIFIER_nondet_pchar` | `char *__VERIFIER_nondet_pchar(void)` | `map2check_gen_data_fuzzer_pchar` | new | Feature `libfuzzer`. Core target of `map2check_non_det_pchar` (#6). An `unsigned` length, NULL for 0, capped at 4096, then `length - 1` chars and a NUL, as `NonDetGeneratorAFL.c:101-117`; the string comes from the C `malloc` and the program may `free` it (SD-18). | +| `__VERIFIER_assume` | `void __VERIFIER_assume(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assume(0)` abandons the current fuzzer input and `LLVMFuzzerTestOneInput` returns 0, as develop's `nondet_cancel`; with no iteration running it calls `exit(0)` (SD-07). Defined `extern "C-unwind"`; see “`__VERIFIER_assume` under a fuzzer”. | +| `map2check_fuzzer_assume` | `void map2check_fuzzer_assume(int32_t condition)` | `map2check_fuzzer_assume(int)` (`nondet_gen_libfuzzer.h:41`) | changed | Feature `libfuzzer`. Same behaviour as `__VERIFIER_assume` (SD-07). | | `__VERIFIER_error` | `void __VERIFIER_error(void)` | — | new | Feature `libfuzzer`. Records REACHABILITY, prints the JSON report when the mode is unset, then aborts (SD-08). | | `__VERIFIER_assert` | `void __VERIFIER_assert(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assert(0)` calls `__VERIFIER_error()` (SD-08). | | `next_current_step` | — | `next_current_step()` | discontinued | Use `get_next_step()`; same effect (#13). | @@ -160,8 +163,8 @@ Map2Check. | `map2check_assume_loop` | — | pending | #21. | | `map2check_crab_assume` | — | pending | #21. | | `map2check_assert` | — | pending | #21. | -| `map2check_non_det_*` | 16 typed generators: bool, char, double, int, loff_t, long, pchar, pointer, sector_t, short, size_t, uchar, uint, ulong, unsigned, ushort | pending | #21; see also #6. | -| `map2check_nondet_*` | `(u32 line, u32 scope, i32 value, ptr fn)`, `double` value for `double` | pending | Core equivalents: `map2check_save_nondet_log_(line, scope, type_var, T *value, fn)` for every type; the pass stores the value in an `alloca` and passes its address (Task 3.2). See “Nondet log”. | +| `map2check_non_det_*` | 16 typed generators: bool, char, double, int, loff_t, long, pchar, pointer, sector_t, short, size_t, uchar, uint, ulong, unsigned, ushort | pending | Core equivalents: `__VERIFIER_nondet_` for every type (`unsigned` → `__VERIFIER_nondet_uint`), reading the input as develop's AFL++ generator (#6, SD-18). The pass stops renaming the call in Task 3.2. | +| `map2check_nondet_*` | `(u32 line, u32 scope, i32 value, ptr fn)`, `double` value for `double` | pending | Core equivalents: `map2check_save_nondet_log_(line, scope, type_var, T *value, fn)` for every type; the pass stores the value in an `alloca` and passes its address (Task 3.2). See “Nondet generators and the nondet log”. | | `map2check_nondet_cast_*` | — | pending | #21. | | `map2check_pointer_cast` | — | pending | #21. | | `map2check_wasm_malloc` | — | out-of-scope | Only used by the `feat-wasm-verification` flow (#21). | @@ -252,14 +255,35 @@ In memtrack and memcleanup modes `map2check_map_malloc` and scenario per verdict, and `rust/src/ffi_golden_tests.rs` checks that this library writes the same bytes for the same calls. -## Nondet log +## Nondet generators and the nondet log -`map2check_save_nondet_log_` keeps the core signature +Task 2.5 (#6) completed the nondet family for the Map2Check passes. Every +`map2check_non_det_` the pass emits has a core `__VERIFIER_nondet_` +target, `pchar`, `loff_t` and `sector_t` included, and every +`map2check_nondet_` logger has a `map2check_save_nondet_log_` +(`docs/cutover/abi-mapping.tsv`). The earlier decision not to port the +`pchar`/`loff_t`/`sector_t` generators is reversed: the pass does emit +them (`NonDetPass.cpp:155-171`). + +**Generators.** Each reads the fuzzer input as develop's AFL++ generator +does (`NonDetGeneratorAFL.c`), which D1 conditions 4 and 9 require so that +CmpLog sees input bytes in the values and a KLEE vector means the same +thing as an AFL++ seed: `sizeof(type)` bytes per value, in order, little-endian +on x86-64, rewinding at the end of the input (SD-18). + +| Type | C type (develop) | Bytes | +|---|---|---| +| `char`, `uchar` | `char`, `unsigned char` | 1 | +| `short`, `ushort` | `short`, `unsigned short` | 2 | +| `int`, `uint`/`unsigned`, `bool` | `int`, `unsigned`, `int` (`typedef int bool`) | 4 | +| `long`, `ulong`, `size_t`, `loff_t`, `sector_t`, `pointer`, `double` | `long`, `unsigned long`, `size_t`, `long long`, `unsigned long`, `void *`, `double` | 8 | +| `pchar` | `char *` | 4 (length) + `length - 1` | + +**Nondet log.** `map2check_save_nondet_log_` keeps the core signature `(int line, int scope, int type_var, T *value, const char *function_name)` -returning `non_det_log_t *`, with `T` the C type develop's -`NonDetGenerator.h` gives the family (`bool` is `int`, `loff_t` is -`long long`, `sector_t` is `unsigned long`; `pointer` and `pchar` pass a -`void **` and a `char **`), the decision of #13. The pass stores the call's result in an `alloca` of that type and +returning `non_det_log_t *`, with `T` the C type of the table above +(`pointer` and `pchar` pass a `void **` and a `char **`), the decision of +#13. The pass stores the call's result in an `alloca` of that type and passes its address; for `bool` it widens the `_Bool` to `int` first. The `klee_log.csv` row is the one develop writes: the `enum NONDET_TYPE` number of the export (`Map2CheckTypes.h:41-58`), and the value as @@ -269,25 +293,28 @@ printed with `%u` for `unsigned`, `%lf` for `double` and `%d` otherwise (`NonDetLog.c:45-54`). The golden `tests/legacy_golden/NONDET-TYPES/` holds one row of each type. `type_var` is only kept in the returned record. -## Discontinued: `map2check_gen_data_fuzzer_pchar` / `_loff_t` / `_sector_t` - -**Status:** discontinued, not ported to `feature/rust`. - -The legacy C library exposed LibFuzzer-backed nondet generators for a few -Linux-kernel/POSIX-specific types (`pchar`, `loff_t`, `sector_t`) alongside -the generic numeric ones. `feature/rust` replaces the whole -`map2check_gen_data_fuzzer_*` family with the SV-COMP-standard -`__VERIFIER_nondet_*` functions (`rust/src/nondet/libfuzzer.rs`), which cover -every portable numeric/bool/pointer type. `pchar`/`loff_t`/`sector_t` have no -portable Rust equivalent (they are kernel ABI types, not part of any target -this library instruments), and no consumer in this repository or in -`Map2Check@develop`'s passes references them. Reintroducing them would mean -adding non-portable, untested surface area for a use case nobody currently -exercises. - -**Decision:** do not implement. If a concrete consumer appears, add a -`__VERIFIER_nondet_*`-style generator for the specific width needed instead -of resurrecting the old kernel-typed names. +## `__VERIFIER_assume` under a fuzzer + +`__VERIFIER_assume(0)` and `map2check_fuzzer_assume(0)` abandon the current +input, as develop's `nondet_cancel` does with `longjmp` +(`NonDetGeneratorAFL.c:31-37`, `:184-196`), and the fuzzer continues with +the next input in the same process (SD-07). + +The mechanism is an unwind, not `longjmp`. Rust cannot call `setjmp` +soundly (it returns twice), and a `longjmp` over Rust frames is undefined +when one of them owns a destructor, as `LLVMFuzzerTestOneInput` does. The +assume calls `std::panic::resume_unwind` with a private payload (no panic +hook runs, nothing is printed), and `LLVMFuzzerTestOneInput` catches it with +`catch_unwind` and returns 0. The exports and the `__map2check_main__` +import are `extern "C-unwind"`, the ABI that lets a Rust unwind cross C +frames (RFC 2945). The analysed program's frames in between must carry +unwind tables, which clang emits by default on x86-64 Linux; they hold no +cleanups, so the unwind skips exactly what develop's `longjmp` skips. The +unwind needs `panic = "unwind"` (the `release` and `runtime` profiles). + +With no iteration running (no `LLVMFuzzerTestOneInput` on the stack), or in +a `panic = "abort"` build, the assume keeps ending the process with +`exit(0)`, the SV-COMP convention for an infeasible path. ## Accepted: failure behaviour at the FFI boundary (panic strategy) @@ -300,26 +327,12 @@ Every `#[no_mangle]` export runs its body through `ffi_guard` | `dev`, `release`, `runtime` | `unwind` | Caught by `catch_unwind` in `ffi_guard`; a diagnostic goes to stderr and the export returns its neutral default. Nothing unwinds into C. | | any `panic = "abort"` build (future KLEE build) | `abort` | No recovery: the process aborts at the panic. `ffi_guard` is then a no-op wrapper. | -Deliberate termination (`__VERIFIER_error` aborts, `__VERIFIER_assume(0)` -exits with 0) is not a failure and is unaffected. `tests/ffi_harness.c` +Deliberate termination (`__VERIFIER_error` aborts; `__VERIFIER_assume(0)` +abandons the fuzzer input, or exits with 0 outside one) is not a failure and +is unaffected. `tests/ffi_harness.c` calls every export before `map2check_init()` with valid boundary inputs under ASan/UBSan. -## Accepted: `__VERIFIER_assume` / `map2check_fuzzer_assume` end the process - -`__VERIFIER_assume(0)` and `map2check_fuzzer_assume(0)` call -`std::process::exit(0)` instead of unwinding back to the caller. LibFuzzer -runs many inputs within a single process, and the assumption is violated -arbitrarily deep in a call stack that began at `LLVMFuzzerTestOneInput`. -There is no portable way to unwind out of it: the entry point is -`extern "C"`, and a Rust panic must not cross an FFI boundary regardless of -the release profile's panic strategy. This matches the -convention used by SV-COMP's native/concrete-execution harnesses. It does end -the fuzzing campaign for the current process — accepted as a known -limitation; full symbolic-execution semantics (path pruning under KLEE) is -deferred to the M4 KLEE spike. See doc comments in -`rust/src/nondet/libfuzzer.rs`. - ## Accepted: `map2check_map_non_static_alloca` / `map2check_map_funct_address` semantics The legacy C headers that defined these symbols diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index d736580..422d1ea 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -234,23 +234,39 @@ map2check_reset(); map2check_check_free("p", (void *)0x5000, 0, 4, "main"); /* MemsafetyFree */ ``` -### SD-07 — `__VERIFIER_assume(0)` ends the process +### SD-07 — `__VERIFIER_assume(0)` abandons the fuzzer input; outside one it exits with 0 -**Classification:** accepted limitation -**Tests:** `rust/src/nondet/libfuzzer.rs::verifier_assume_false_exits_with_zero`, `rust/src/nondet/libfuzzer.rs::map2check_fuzzer_assume_false_exits_with_zero` - -**Rust:** `__VERIFIER_assume(0)` and `map2check_fuzzer_assume(0)` call -`exit(0)`. A Rust panic must not unwind across `extern "C"`, so there is no -safe way to return to `LLVMFuzzerTestOneInput`. The cost is that the -current fuzzing process ends. See “Accepted: `__VERIFIER_assume` / -`map2check_fuzzer_assume` end the process” in `api-deltas.md`. - -**Legacy:** not verified against source (#15). +**Classification:** intentional +**Tests:** `rust/src/nondet/libfuzzer.rs::failed_assume_abandons_only_the_current_iteration`, `rust/src/nondet/libfuzzer.rs::failed_fuzzer_assume_abandons_the_iteration_too`, `rust/src/nondet/libfuzzer.rs::verifier_assume_false_exits_with_zero`, `rust/src/nondet/libfuzzer.rs::map2check_fuzzer_assume_false_exits_with_zero` + +**Rust:** inside `LLVMFuzzerTestOneInput`, `__VERIFIER_assume(0)` and +`map2check_fuzzer_assume(0)` abandon the current input: they unwind (a +`resume_unwind` with a private payload) through the analysed program's +frames back to `LLVMFuzzerTestOneInput`, which catches it and returns 0. +The fuzzer goes on with the next input in the same process, and the next +iteration starts from an empty state (SD-11). Both exports, and the +`__map2check_main__` import, are `extern "C-unwind"`; the program's frames +need unwind tables, the clang default on x86-64 Linux. With no iteration +running (a program linked without a fuzzer driver), or in a +`panic = "abort"` build, they keep calling `exit(0)`. + +**Legacy (Map2Check `develop`):** `nondet_assume` calls `nondet_cancel`, +which `longjmp`s back to the `setjmp` in the `__AFL_LOOP` of `main`, and the +loop moves on to the next input (`NonDetGeneratorAFL.c:31-37`, `:184-196`). +The Rust library does the same with an unwind instead of `longjmp`: Rust +cannot call `setjmp`, and a `longjmp` over Rust frames is undefined when +one of them has a destructor. Outside the `__AFL_LOOP` develop has no +case: its `main` always runs the program inside the loop. **Reproduce (link with `--features libfuzzer`):** ```c -__VERIFIER_assume(0); /* process exits with status 0 */ +int __map2check_main__(void) { + __VERIFIER_assume(__VERIFIER_nondet_int() == 7); + /* reached only for inputs that start with 07 00 00 00 */ + return 0; +} +/* LLVMFuzzerTestOneInput returns 0 for every input; the process goes on */ ``` ### SD-08 — `__VERIFIER_error()` reports REACHABILITY, then aborts @@ -534,10 +550,8 @@ appends before it checks (`AnalysisModeMemtrack.c:346-358`); the golden writes it only with the AFL++ or KLEE generator; `NonDetGeneratorNone.c` writes none (`:42`). A `float` value, which develop has no logger for, is logged as a double (type 15). Every other type matches develop byte for - byte, value format and type number included: the value is the one - `NonDetPass.cpp` hands to develop's `int` wrapper (narrow types - zero-extended, so a `char` -3 is `253`; wide types and addresses - truncated to 32 bits), as the golden `NONDET-TYPES` checks. + byte, value format and type number included (golden `NONDET-TYPES`, + SD-18). - **Long function names.** Develop copies names into `char[35]` with `strncpy` (`FUNCTION_MAX_LENGTH_NAME`), which leaves names of 35 or more characters unterminated; this library writes the whole name. @@ -553,3 +567,45 @@ map2check_init(); map2check_binop_add_signed(INT64_MAX, 1, 64, 1, "main"); /* klee_log.csv is written; develop's NonDetGeneratorNone.c writes none */ ``` + +### SD-18 — Nondet values: develop's AFL++ input layout and log format, with three differences + +**Classification:** intentional +**Tests:** `rust/src/nondet/libfuzzer.rs::every_generator_reads_sizeof_type_bytes_and_wraps_like_develop`, `rust/src/nondet/libfuzzer.rs::seed_a_reads_0x41414141_like_develop`, `rust/src/nondet/libfuzzer.rs::bool_consumes_four_bytes_and_is_the_int_converted_to_bool`, `rust/src/nondet/libfuzzer.rs::empty_input_yields_zero`, `rust/src/nondet/libfuzzer.rs::without_an_iteration_every_generator_yields_zero`, `rust/src/nondet/libfuzzer.rs::pchar_reads_an_unsigned_length_then_length_minus_one_chars`, `rust/src/ffi_golden_tests.rs::nondet_types_match_c_runtime` + +**Rust:** the `__VERIFIER_nondet_` generators read the fuzzer input as +develop's `NonDetGeneratorAFL.c` does: `sizeof(type)` bytes per value, in +order, copied in memory order (little-endian on x86-64; `:67-99`), and past +the end of the input the reader rewinds, returning `data[0]` once without +advancing (`get_next_input_from_afl`, `:56-63`). With the 1-byte seed `A` +an `int` is `0x41414141`, as in develop. `pchar` reads an `unsigned` +length, returns NULL for 0, caps it at 4096 and reads `length - 1` chars +(`:101-117`); the string comes from the C `malloc`. The +`map2check_save_nondet_log_` exports write the `klee_log.csv` row +develop writes for the same program: the type number of `enum NONDET_TYPE` +and the value as `NonDetPass.cpp` hands it to the C wrapper (zero-extended +narrow types, so a `char` -3 is `253` and a `short` -2 is `65534`; the low +32 bits of wide types and addresses; `unsigned` with `%u`, `uint` with +`%d`). The differences: + +- **Empty input.** Develop reads `data[0]` of an empty buffer, out of + bounds; the library returns 0 bytes. With no iteration running every + generator returns 0 and `pchar` returns NULL. +- **`bool`.** Develop's `map2check_non_det_bool` returns the raw `int` of + its four bytes to a caller that declared `_Bool`, so what the program + sees depends on how the caller's code reads the register. The library + reads the same four bytes and returns that `int` converted to `_Bool` + (`!= 0`). +- **`float`.** Develop has no `float` generator or logger. The library + reads four bytes for it and logs it as a double (SD-17). + +**Reproduce (link with `--features libfuzzer`):** + +```c +int __map2check_main__(void) { + int x = __VERIFIER_nondet_int(); /* input "A": x == 0x41414141 */ + char *s = __VERIFIER_nondet_pchar(); + free(s); + return 0; +} +``` diff --git a/rust/src/nondet/libfuzzer.rs b/rust/src/nondet/libfuzzer.rs index 2593b72..552f6ca 100644 --- a/rust/src/nondet/libfuzzer.rs +++ b/rust/src/nondet/libfuzzer.rs @@ -3,24 +3,52 @@ // LibFuzzer nondeterministic value generation. // Replaces nondet_gen_libfuzzer.c — eliminates all union type-punning // by using f32::from_bits() / f64::from_bits() from safe Rust. - -use std::sync::atomic::{AtomicPtr, AtomicUsize, Ordering}; +// +// Input semantics are those of Map2Check develop's AFL++ generator +// (`modules/backend/library/lib/NonDetGeneratorAFL.c`), which D1 conditions +// 4 and 9 require (docs/cutover/decisoes.md): +// +// - each value takes `sizeof(type)` bytes of the input, in order, copied +// into the value in memory order (`get_bytes_from_afl`, :67-74, and the +// generators, :76-99): little-endian on x86-64, the layout KLEE writes +// into a `.ktest` for `klee_make_symbolic(&v, sizeof v)`; +// - past the end of the input the reader rewinds (`get_next_input_from_afl`, +// :56-63): the read that finds the index at the end returns `data[0]` +// without advancing, so after every wrap `data[0]` is read twice; +// - `pchar` reads an `unsigned` length, returns NULL for 0, caps it at 4096, +// and fills `length - 1` chars plus a NUL (:101-117); +// - a failed `__VERIFIER_assume` abandons the current input and the loop +// moves on to the next one (`nondet_cancel`/`setjmp`, :31-37, :184-196). + +use std::sync::atomic::{AtomicBool, AtomicPtr, AtomicUsize, Ordering}; static FUZZER_CURSOR: AtomicUsize = AtomicUsize::new(0); static FUZZER_SIZE: AtomicUsize = AtomicUsize::new(0); static FUZZER_DATA: AtomicPtr = AtomicPtr::new(std::ptr::null_mut()); +/// True while `LLVMFuzzerTestOneInput` runs the program for one input. +static ITERATION_ACTIVE: AtomicBool = AtomicBool::new(false); struct FuzzerStateReset; impl Drop for FuzzerStateReset { fn drop(&mut self) { + ITERATION_ACTIVE.store(false, Ordering::SeqCst); FUZZER_DATA.store(std::ptr::null_mut(), Ordering::SeqCst); FUZZER_SIZE.store(0, Ordering::SeqCst); FUZZER_CURSOR.store(0, Ordering::SeqCst); } } -/// Called by LibFuzzer before the test target runs. +/// Unwind payload of a failed `__VERIFIER_assume` inside an iteration: the +/// Rust counterpart of develop's `longjmp(map2check_reject_env, 1)`. +struct InputRejected; + +/// Called by LibFuzzer (or AFL++'s libFuzzer driver) once per input. +/// +/// A failed `__VERIFIER_assume` ends the iteration early, as develop's +/// `nondet_cancel` does: it unwinds to this function, which returns 0 so the +/// fuzzer goes on with the next input. The analysis state is reset at the +/// start of every iteration (#9), so the abandoned one leaves nothing behind. /// /// # Safety /// `data` must remain valid for the duration of one fuzzer iteration. @@ -28,140 +56,144 @@ impl Drop for FuzzerStateReset { #[no_mangle] pub unsafe extern "C" fn LLVMFuzzerTestOneInput(data: *const u8, size: usize) -> i32 { let _reset = FuzzerStateReset; - crate::ffi::ffi_guard( + let started = crate::ffi::ffi_guard( || { crate::state::reset(); - if data.is_null() && size != 0 { - return 0; + return false; } - // SAFETY: LibFuzzer guarantees `data` is valid for `size` bytes. FUZZER_DATA.store(data as *mut u8, Ordering::SeqCst); FUZZER_SIZE.store(size, Ordering::SeqCst); FUZZER_CURSOR.store(0, Ordering::SeqCst); - - unsafe extern "C" { - fn __map2check_main__() -> i32; - } - // SAFETY: user-defined entry point, linked by Map2Check instrumentation. - unsafe { __map2check_main__() } + ITERATION_ACTIVE.store(true, Ordering::SeqCst); + true }, - 0, - ) -} - -fn read_bytes(n: usize) -> Option<&'static [u8]> { - let cursor = FUZZER_CURSOR.load(Ordering::SeqCst); - let size = FUZZER_SIZE.load(Ordering::SeqCst); - let end = match cursor.checked_add(n) { - Some(end) if end <= size => end, - _ => return None, - }; - - let ptr = FUZZER_DATA.load(Ordering::SeqCst); - if ptr.is_null() { - return None; + false, + ); + if !started { + return 0; } - - // SAFETY: FUZZER_DATA was set from a valid LibFuzzer buffer; cursor+n <= size. - let slice = unsafe { std::slice::from_raw_parts(ptr.add(cursor), n) }; - FUZZER_CURSOR.store(end, Ordering::SeqCst); - Some(slice) -} - -fn read_u8() -> u8 { - read_bytes(1).map(|b| b[0]).unwrap_or(0) -} - -fn read_u32() -> u32 { - read_bytes(4) - .map(|b| u32::from_le_bytes(b.try_into().unwrap_or([0; 4]))) - .unwrap_or(0) + unsafe extern "C-unwind" { + /// The analysed program's `main`, renamed by the Map2Check pass. + /// `C-unwind`: a failed `__VERIFIER_assume` unwinds out of it. + fn __map2check_main__() -> i32; + } + // SAFETY: user-defined entry point, linked by Map2Check instrumentation. + let run = std::panic::catch_unwind(|| unsafe { __map2check_main__() }); + if let Err(payload) = run { + if !payload.is::() { + eprintln!("[map2check] panic in FFI boundary; recovering safely"); + } + } + 0 } -fn read_u64() -> u64 { - read_bytes(8) - .map(|b| u64::from_le_bytes(b.try_into().unwrap_or([0; 8]))) - .unwrap_or(0) +/// Fills `N` bytes from the input as develop's `get_bytes_from_afl` does, +/// one `get_next_input_from_afl` per byte. With no input (no iteration, or an +/// empty one) every byte is 0: develop would read `data[0]` of an empty +/// buffer, out of bounds. +fn read_value() -> [u8; N] { + let mut out = [0_u8; N]; + let ptr = FUZZER_DATA.load(Ordering::Relaxed); + let size = FUZZER_SIZE.load(Ordering::Relaxed); + if ptr.is_null() || size == 0 { + return out; + } + // SAFETY: FUZZER_DATA was set from a valid LibFuzzer buffer of `size` + // bytes, which stays valid for the whole iteration. + let data = unsafe { std::slice::from_raw_parts(ptr, size) }; + let mut index = FUZZER_CURSOR.load(Ordering::Relaxed); + for byte in &mut out { + if index < size { + *byte = data[index]; + index += 1; + } else { + index = 0; + *byte = data[0]; + } + } + FUZZER_CURSOR.store(index, Ordering::Relaxed); + out } // ── Public nondet generators ───────────────────────────────────────────────── +/// develop's `bool` is `typedef int bool` (`NonDetGenerator.h:74`): four +/// bytes, returned as that `int` converted to `_Bool`. #[no_mangle] pub extern "C" fn __VERIFIER_nondet_bool() -> bool { - crate::ffi::ffi_guard(|| read_u8() % 2 == 0, false) + crate::ffi::ffi_guard(|| i32::from_ne_bytes(read_value()) != 0, false) } #[no_mangle] pub extern "C" fn __VERIFIER_nondet_char() -> i8 { - crate::ffi::ffi_guard(|| read_u8() as i8, 0) + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) } #[no_mangle] pub extern "C" fn __VERIFIER_nondet_uchar() -> u8 { - crate::ffi::ffi_guard(read_u8, 0) + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) } #[no_mangle] pub extern "C" fn __VERIFIER_nondet_short() -> i16 { - crate::ffi::ffi_guard( - || { - read_bytes(2) - .map(|b| i16::from_le_bytes(b.try_into().unwrap_or([0; 2]))) - .unwrap_or(0) - }, - 0, - ) + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) } #[no_mangle] pub extern "C" fn __VERIFIER_nondet_ushort() -> u16 { - crate::ffi::ffi_guard( - || { - read_bytes(2) - .map(|b| u16::from_le_bytes(b.try_into().unwrap_or([0; 2]))) - .unwrap_or(0) - }, - 0, - ) + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) } #[no_mangle] pub extern "C" fn __VERIFIER_nondet_int() -> i32 { - crate::ffi::ffi_guard(|| read_u32() as i32, 0) + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) } #[no_mangle] pub extern "C" fn __VERIFIER_nondet_uint() -> u32 { - crate::ffi::ffi_guard(read_u32, 0) + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) } #[no_mangle] pub extern "C" fn __VERIFIER_nondet_long() -> i64 { - crate::ffi::ffi_guard(|| read_u64() as i64, 0) + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) } #[no_mangle] pub extern "C" fn __VERIFIER_nondet_ulong() -> u64 { - crate::ffi::ffi_guard(read_u64, 0) + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) +} + +#[no_mangle] +pub extern "C" fn __VERIFIER_nondet_size_t() -> usize { + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) +} + +/// glibc's `loff_t` is `long long`. +#[no_mangle] +pub extern "C" fn __VERIFIER_nondet_loff_t() -> i64 { + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) } +/// develop: `#define sector_t unsigned long` (`NonDetGenerator.h:112`). +#[no_mangle] +pub extern "C" fn __VERIFIER_nondet_sector_t() -> std::os::raw::c_ulong { + crate::ffi::ffi_guard(|| ::from_ne_bytes(read_value()), 0) +} + +/// Four bytes, like `int`; develop has no `float` generator. #[no_mangle] pub extern "C" fn __VERIFIER_nondet_float() -> f32 { // Uses f32::from_bits — replaces C union FloatPattern type-punning safely - crate::ffi::ffi_guard(|| f32::from_bits(read_u32()), 0.0) + crate::ffi::ffi_guard(|| f32::from_bits(u32::from_ne_bytes(read_value())), 0.0) } #[no_mangle] pub extern "C" fn __VERIFIER_nondet_double() -> f64 { // Uses f64::from_bits — replaces C union DoublePattern type-punning safely - crate::ffi::ffi_guard(|| f64::from_bits(read_u64()), 0.0) -} - -#[no_mangle] -pub extern "C" fn __VERIFIER_nondet_size_t() -> usize { - crate::ffi::ffi_guard(|| read_u64() as usize, 0) + crate::ffi::ffi_guard(|| f64::from_bits(u64::from_ne_bytes(read_value())), 0.0) } #[no_mangle] @@ -169,48 +201,104 @@ pub extern "C" fn __VERIFIER_nondet_pointer() -> *mut std::ffi::c_void { // Constructing a pointer value from an integer is safe; only // dereferencing it would be unsafe, and nothing here dereferences it. crate::ffi::ffi_guard( - || read_u64() as usize as *mut std::ffi::c_void, + || usize::from_ne_bytes(read_value()) as *mut std::ffi::c_void, std::ptr::null_mut(), ) } -// ── SV-COMP verifier primitives ────────────────────────────────────────────── -// -// Design note (documented per the project convention of registering -// trade-offs where they are made): LibFuzzer runs many inputs within a -// single process, and an assumption is violated arbitrarily deep in a call -// stack that began at `LLVMFuzzerTestOneInput`. There is no portable way to -// unwind out of it: the entry point is `extern "C"`, and a Rust panic must -// not cross an FFI boundary regardless of the release profile's panic -// strategy, so an unwind-based `assume` would be unsound rather than merely -// awkward. Following the same convention used by SV-COMP's -// native/concrete-execution harnesses, `__VERIFIER_assume` ends the current -// process with exit code 0 (treated as "uninteresting", not a crash) instead -// of unwinding. This does end the fuzzing campaign for the current process — -// an accepted limitation, see docs/api-deltas.md. Full symbolic-execution -// semantics (e.g. path pruning under KLEE) is out of scope for M3 and is -// deferred to the M4 KLEE spike. - -/// Ends the process with exit code 0 (treated as "uninteresting", not a -/// crash) when `condition` is false. See the design note above. +/// develop's `MAP2CHECK_MAX_FUZZED_STRING` (`NonDetGeneratorAFL.c:101`). +const MAX_FUZZED_STRING: u32 = 4096; + +/// A NUL-terminated string from the input, allocated with the C `malloc` so +/// the program can `free` it (`NonDetGeneratorAFL.c:103-117`): an +/// `unsigned` length, NULL when it is 0 (or when `malloc` fails), capped at +/// 4096, then `length - 1` chars, each read like `__VERIFIER_nondet_char`. #[no_mangle] -pub extern "C" fn __VERIFIER_assume(condition: i32) { +pub extern "C" fn __VERIFIER_nondet_pchar() -> *mut std::os::raw::c_char { + extern "C" { + fn malloc(size: usize) -> *mut std::ffi::c_void; + } crate::ffi::ffi_guard( || { - if condition == 0 { - std::process::exit(0); + let length = u32::from_ne_bytes(read_value()).min(MAX_FUZZED_STRING) as usize; + if length == 0 { + return std::ptr::null_mut(); + } + // SAFETY: plain C allocation; the result is checked for NULL. + let string = unsafe { malloc(length) }.cast::(); + if string.is_null() { + return std::ptr::null_mut(); + } + // SAFETY: `string` holds `length` bytes, written once each below. + let bytes = unsafe { std::slice::from_raw_parts_mut(string, length) }; + let (chars, nul) = bytes.split_at_mut(length - 1); + for c in chars { + *c = u8::from_ne_bytes(read_value()); } + nul[0] = 0; + string.cast() }, - (), - ); + std::ptr::null_mut(), + ) +} + +// ── SV-COMP verifier primitives ────────────────────────────────────────────── +// +// `__VERIFIER_assume(0)` inside a fuzzer iteration abandons that input, as +// develop's `nondet_assume` → `nondet_cancel` → `longjmp` back to the +// `__AFL_LOOP` does (NonDetGeneratorAFL.c:31-37, :184-196): no exit, no +// abort, and the next input runs in the same process. +// +// Mechanism (design note): a `longjmp` from Rust back to a `setjmp` would +// need the `setjmp` in C (Rust cannot call `setjmp` soundly, it returns +// twice) and would skip Rust frames, which is only defined when none of +// them has a destructor; `LLVMFuzzerTestOneInput` owns a drop guard. Instead +// the assume unwinds with `std::panic::resume_unwind` (no panic hook, no +// message) and `LLVMFuzzerTestOneInput` catches it with `catch_unwind`. +// Every Rust function on that path is declared `C-unwind` +// (`__VERIFIER_assume`, `map2check_fuzzer_assume`, the `__map2check_main__` +// import), which is the defined way for a Rust unwind to cross C frames +// (RFC 2945). The C frames in between are the analysed program's: they +// must carry unwind tables, which clang emits by default on x86-64 Linux +// (`-funwind-tables`); they hold no cleanups, so nothing is skipped that +// develop's `longjmp` would not skip as well. The unwind needs +// `panic = "unwind"` (the `runtime` and `release` profiles); built with +// `panic = "abort"` the library keeps the exit(0) fallback below. +// +// Outside an iteration (no `LLVMFuzzerTestOneInput` on the stack, e.g. a +// program that links the library without a fuzzer driver), +// `__VERIFIER_assume(0)` keeps ending the process with exit code 0, the +// SV-COMP convention for an infeasible path (SD-07). + +/// Abandons the current fuzzer input, or ends the process with exit code 0 +/// when there is none. Never returns. +fn reject_input() -> ! { + #[cfg(panic = "unwind")] + if ITERATION_ACTIVE.load(Ordering::SeqCst) { + std::panic::resume_unwind(Box::new(InputRejected)); + } + std::process::exit(0); +} + +/// Abandons the current fuzzer input when `condition` is false (see the +/// design note above); outside an iteration, ends the process with exit 0. +#[no_mangle] +pub extern "C-unwind" fn __VERIFIER_assume(condition: i32) { + // The decision runs under ffi_guard; the unwind must not, or ffi_guard + // would catch it. + if crate::ffi::ffi_guard(|| condition == 0, false) { + reject_input(); + } } /// Fuzzer-specific alias of `__VERIFIER_assume`, matching the legacy /// `map2check_fuzzer_assume` entry point used by the LibFuzzer nondet -/// generator. See `__VERIFIER_assume` for the exit(0) design note. +/// generator. #[no_mangle] -pub extern "C" fn map2check_fuzzer_assume(condition: i32) { - crate::ffi::ffi_guard(|| __VERIFIER_assume(condition), ()); +pub extern "C-unwind" fn map2check_fuzzer_assume(condition: i32) { + if crate::ffi::ffi_guard(|| condition == 0, false) { + reject_input(); + } } /// SV-COMP reachability primitive: records a `Reachability` violation and @@ -256,7 +344,7 @@ pub extern "C" fn __VERIFIER_assert(condition: i32) { // Cria uma função "falsa" apenas para os testes passarem no Linker #[cfg(test)] #[no_mangle] -pub extern "C" fn __map2check_main__() -> i32 { +pub extern "C-unwind" fn __map2check_main__() -> i32 { // Runs the per-test hook (if any); otherwise succeeds and does nothing. let hook = *TEST_MAIN_HOOK .lock() @@ -313,57 +401,293 @@ mod tests { f() } + // ── Input byte layout (D1 conditions 4 and 9) ─────────────────────────── + // + // develop's NonDetGeneratorAFL.c copies sizeof(type) bytes per value, in + // memory order, into the value (`get_bytes_from_afl`, :67-74; the + // generators, :76-99), so a value is the little-endian reading of its + // bytes on x86-64. Past the end of the input `get_next_input_from_afl` + // (:56-63) rewinds: the read that finds the index at the end returns + // data[0] WITHOUT advancing, so data[0] is read twice after every wrap. + + /// develop's `get_next_input_from_afl`, transcribed: the bytes `count` + /// reads return from a fresh input. + fn develop_stream(data: &[u8], count: usize) -> Vec { + let mut index = 0; + (0..count) + .map(|_| { + if index < data.len() { + index += 1; + data[index - 1] + } else { + index = 0; + data[0] + } + }) + .collect() + } + #[test] - fn narrow_generators_decode_little_endian_input_in_order() { - let data = [ - 0x02, // bool: 2 % 2 == 0 -> true - 0xff, // char: -1 - 0xfe, // uchar: 254 - 0x01, 0x80, // short: 0x8001 -> -32767 - 0x34, 0x12, // ushort: 0x1234 - 0xff, 0xff, 0xff, 0xff, // int: -1 - 0x78, 0x56, 0x34, 0x12, // uint: 0x1234_5678 - ]; - with_input(&data, || { - assert!(__VERIFIER_nondet_bool()); - assert_eq!(__VERIFIER_nondet_char(), -1); - assert_eq!(__VERIFIER_nondet_uchar(), 254); - assert_eq!(__VERIFIER_nondet_short(), -32767); - assert_eq!(__VERIFIER_nondet_ushort(), 0x1234); - assert_eq!(__VERIFIER_nondet_int(), -1); - assert_eq!(__VERIFIER_nondet_uint(), 0x1234_5678); + fn develop_stream_model_rewinds_and_repeats_the_first_byte() { + // Hand-checked against NonDetGeneratorAFL.c:56-63. + assert_eq!(develop_stream(&[1, 2], 7), [1, 2, 1, 1, 2, 1, 1]); + assert_eq!(develop_stream(&[0x41], 4), [0x41; 4]); + } + + /// Reads two values with `$gen` from `data` and checks them against the + /// bytes develop's reader returns for the same input. + macro_rules! layout_case { + ($gen:ident, $ty:ty, $data:expr) => {{ + let data: &[u8] = $data; + let size = std::mem::size_of::<$ty>(); + with_input(data, || { + let first: $ty = $gen(); + let second: $ty = $gen(); + let stream = develop_stream(data, 2 * size); + let expect_first = <$ty>::from_ne_bytes(stream[..size].try_into().unwrap()); + let expect_second = <$ty>::from_ne_bytes(stream[size..].try_into().unwrap()); + assert_eq!( + first.to_ne_bytes(), + expect_first.to_ne_bytes(), + "{}: primeiro valor de {data:02x?}", + stringify!($gen) + ); + assert_eq!( + second.to_ne_bytes(), + expect_second.to_ne_bytes(), + "{}: segundo valor de {data:02x?}", + stringify!($gen) + ); + }); + }}; + } + + /// Short input (fewer bytes than the type), exact-size input, and a + /// second value that spans the wrap, for one generator. + macro_rules! layout_cases { + ($gen:ident, $ty:ty) => {{ + let size = std::mem::size_of::<$ty>(); + let bytes: Vec = (1..=2 * size as u8 + 3) + .map(|b| b.wrapping_mul(0x11)) + .collect(); + // shorter than the type (one byte when the type has one byte) + layout_case!($gen, $ty, &bytes[..size.saturating_sub(1).max(1)]); + // exactly the type: the second value is all wrap + layout_case!($gen, $ty, &bytes[..size]); + // the second value starts inside the input and wraps + layout_case!($gen, $ty, &bytes[..size + size / 2 + 1]); + }}; + } + + #[test] + fn every_generator_reads_sizeof_type_bytes_and_wraps_like_develop() { + layout_cases!(__VERIFIER_nondet_char, i8); + layout_cases!(__VERIFIER_nondet_uchar, u8); + layout_cases!(__VERIFIER_nondet_short, i16); + layout_cases!(__VERIFIER_nondet_ushort, u16); + layout_cases!(__VERIFIER_nondet_int, i32); + layout_cases!(__VERIFIER_nondet_uint, u32); + layout_cases!(__VERIFIER_nondet_long, i64); + layout_cases!(__VERIFIER_nondet_ulong, u64); + layout_cases!(__VERIFIER_nondet_size_t, usize); + layout_cases!(__VERIFIER_nondet_loff_t, i64); + layout_cases!(__VERIFIER_nondet_sector_t, std::os::raw::c_ulong); + layout_cases!(nondet_float_bits, u32); + layout_cases!(nondet_double_bits, u64); + layout_cases!(nondet_pointer_bits, usize); + } + + fn nondet_float_bits() -> u32 { + __VERIFIER_nondet_float().to_bits() + } + + fn nondet_double_bits() -> u64 { + __VERIFIER_nondet_double().to_bits() + } + + fn nondet_pointer_bits() -> usize { + __VERIFIER_nondet_pointer() as usize + } + + #[test] + fn seed_a_reads_0x41414141_like_develop() { + // The spike's seed (`printf A`): develop reads x = 0x41414141, which + // CmpLog can match against the input (D1 condition 4). + with_input(b"A", || assert_eq!(__VERIFIER_nondet_int(), 0x4141_4141)); + with_input(&[0x2a, 0, 0, 0], || assert_eq!(__VERIFIER_nondet_int(), 42)); + } + + #[test] + fn bool_consumes_four_bytes_and_is_the_int_converted_to_bool() { + with_input(&[0, 0, 0, 0, 0, 1, 0, 0, 7], || { + assert!(!__VERIFIER_nondet_bool()); // 0x00000000 + assert!(__VERIFIER_nondet_bool()); // 0x00000100: low byte 0, int != 0 + assert_eq!(__VERIFIER_nondet_uchar(), 7); // the ninth byte }); + // short inputs wrap: 00 00 00 00 and 00 01 00 00 + with_input(&[0], || assert!(!__VERIFIER_nondet_bool())); + with_input(&[0, 1], || assert!(__VERIFIER_nondet_bool())); } #[test] - fn wide_generators_decode_eight_byte_values() { + fn values_are_little_endian_in_input_order() { let mut data = Vec::new(); data.extend_from_slice(&(-2_i64).to_le_bytes()); - data.extend_from_slice(&u64::MAX.to_le_bytes()); data.extend_from_slice(&1.5_f32.to_bits().to_le_bytes()); data.extend_from_slice(&(-2.25_f64).to_bits().to_le_bytes()); - data.extend_from_slice(&4096_u64.to_le_bytes()); data.extend_from_slice(&0x1000_u64.to_le_bytes()); + data.extend_from_slice(&(-32767_i16).to_le_bytes()); with_input(&data, || { assert_eq!(__VERIFIER_nondet_long(), -2); - assert_eq!(__VERIFIER_nondet_ulong(), u64::MAX); assert_eq!(__VERIFIER_nondet_float().to_bits(), 1.5_f32.to_bits()); assert_eq!(__VERIFIER_nondet_double().to_bits(), (-2.25_f64).to_bits()); - assert_eq!(__VERIFIER_nondet_size_t(), 4096); assert_eq!(__VERIFIER_nondet_pointer() as usize, 0x1000); + assert_eq!(__VERIFIER_nondet_short(), -32767); + }); + } + + #[test] + fn empty_input_yields_zero() { + // develop reads data[0] of an empty buffer (out of bounds); the + // library returns zero bytes instead (semantic-deltas.md). + with_input(&[], || { + assert_eq!(__VERIFIER_nondet_int(), 0); + assert_eq!(__VERIFIER_nondet_ulong(), 0); + assert!(__VERIFIER_nondet_pchar().is_null()); + }); + } + + #[test] + fn without_an_iteration_every_generator_yields_zero() { + let _guard = crate::state::test_lock(); + assert!(!__VERIFIER_nondet_bool()); + assert_eq!(__VERIFIER_nondet_char(), 0); + assert_eq!(__VERIFIER_nondet_uchar(), 0); + assert_eq!(__VERIFIER_nondet_short(), 0); + assert_eq!(__VERIFIER_nondet_ushort(), 0); + assert_eq!(__VERIFIER_nondet_int(), 0); + assert_eq!(__VERIFIER_nondet_uint(), 0); + assert_eq!(__VERIFIER_nondet_long(), 0); + assert_eq!(__VERIFIER_nondet_ulong(), 0); + assert_eq!(__VERIFIER_nondet_size_t(), 0); + assert_eq!(__VERIFIER_nondet_float().to_bits(), 0); + assert_eq!(__VERIFIER_nondet_double().to_bits(), 0); + assert!(__VERIFIER_nondet_pointer().is_null()); + assert_eq!(__VERIFIER_nondet_loff_t(), 0); + assert_eq!(__VERIFIER_nondet_sector_t(), 0); + assert!(__VERIFIER_nondet_pchar().is_null()); + } + + // ── pchar (NonDetGeneratorAFL.c:101-117) ───────────────────────────────── + + extern "C" { + fn free(ptr: *mut std::ffi::c_void); + } + + /// Takes a `__VERIFIER_nondet_pchar` result: its bytes up to the NUL, and + /// frees it with the C allocator, as the analysed program would. + fn take_pchar(p: *mut std::os::raw::c_char) -> Option> { + if p.is_null() { + return None; + } + // SAFETY: a non-null result is a NUL-terminated malloc'd string. + let bytes = unsafe { std::ffi::CStr::from_ptr(p) }.to_bytes().to_vec(); + // SAFETY: allocated with malloc by the generator; freed once. + unsafe { free(p.cast()) }; + Some(bytes) + } + + #[test] + fn pchar_reads_an_unsigned_length_then_length_minus_one_chars() { + with_input(&[3, 0, 0, 0, b'a', b'b', b'c'], || { + assert_eq!(take_pchar(__VERIFIER_nondet_pchar()), Some(b"ab".to_vec())); + // the cursor is at 'c': the next generator reads it + assert_eq!(__VERIFIER_nondet_uchar(), b'c'); + }); + with_input(&[1, 0, 0, 0], || { + assert_eq!(take_pchar(__VERIFIER_nondet_pchar()), Some(Vec::new())); + }); + with_input(&[0, 0, 0, 0], || { + assert!(__VERIFIER_nondet_pchar().is_null()) }); } #[test] - fn exhausted_input_yields_zero_without_advancing() { - with_input(&[0x07], || { - assert_eq!(__VERIFIER_nondet_int(), 0); // needs 4 bytes, only 1 left - assert_eq!(__VERIFIER_nondet_uchar(), 7); // failed read did not consume - assert_eq!(__VERIFIER_nondet_uchar(), 0); - assert!(__VERIFIER_nondet_bool()); // 0 % 2 == 0 + fn pchar_caps_the_length_at_4096_and_wraps_for_its_chars() { + // length 0x41414141 > 4096: capped to 4096, 4095 chars of 'A'. + with_input(b"A", || { + let s = take_pchar(__VERIFIER_nondet_pchar()).unwrap(); + assert_eq!(s.len(), 4095); + assert!(s.iter().all(|&c| c == b'A')); }); } + // ── __VERIFIER_assume inside an iteration (develop: nondet_cancel) ─────── + + static AFTER_ASSUME: AtomicUsize = AtomicUsize::new(0); + + /// Fake program: tracks a block, rejects inputs whose first int is not + /// 7 (as `nondet_assume` does with `longjmp`), and counts the runs that + /// got past the assume. + fn assume_body() { + // SAFETY: fake address, only stored. + unsafe { crate::ffi::map2check_map_malloc(0x2000 as *const _, 8) }; + let x = __VERIFIER_nondet_int(); + __VERIFIER_assume(i32::from(x == 7)); + AFTER_ASSUME.fetch_add(1, Ordering::SeqCst); + } + + #[test] + fn failed_assume_abandons_only_the_current_iteration() { + let _guard = crate::state::test_lock(); + set_hook(Some(assume_body)); + AFTER_ASSUME.store(0, Ordering::SeqCst); + + let rejected = 1_i32.to_le_bytes(); + let accepted = 7_i32.to_le_bytes(); + // SAFETY: buffers are valid for their lengths for the whole call. + let rc = unsafe { LLVMFuzzerTestOneInput(rejected.as_ptr(), rejected.len()) }; + assert_eq!(rc, 0); + assert_eq!(AFTER_ASSUME.load(Ordering::SeqCst), 0, "assume não cortou"); + assert!(!ITERATION_ACTIVE.load(Ordering::SeqCst)); + assert!(FUZZER_DATA.load(Ordering::SeqCst).is_null()); + // The state is usable at once, as at the start of an iteration. + assert!(crate::state::with_state(|s| Ok(s.result.ok)).is_ok()); + + // SAFETY: as above. + let rc = unsafe { LLVMFuzzerTestOneInput(accepted.as_ptr(), accepted.len()) }; + set_hook(None); + assert_eq!(rc, 0); + assert_eq!(AFTER_ASSUME.load(Ordering::SeqCst), 1); + crate::state::with_state(|s| { + assert_eq!(s.memtrack.len(), 1, "o bloco da iteração rejeitada vazou"); + Ok(()) + }) + .unwrap(); + } + + fn fuzzer_assume_body() { + map2check_fuzzer_assume(0); + AFTER_ASSUME.fetch_add(1, Ordering::SeqCst); + } + + #[test] + fn failed_fuzzer_assume_abandons_the_iteration_too() { + let _guard = crate::state::test_lock(); + set_hook(Some(fuzzer_assume_body)); + AFTER_ASSUME.store(0, Ordering::SeqCst); + let data = [0_u8; 4]; + for _ in 0..3 { + // SAFETY: valid buffer for the whole call. + assert_eq!( + unsafe { LLVMFuzzerTestOneInput(data.as_ptr(), data.len()) }, + 0 + ); + } + set_hook(None); + assert_eq!(AFTER_ASSUME.load(Ordering::SeqCst), 0); + } + #[test] fn null_buffer_with_nonzero_size_yields_zero() { let _guard = crate::state::test_lock(); diff --git a/test/codefuzz_by_caller_test.c b/test/codefuzz_by_caller_test.c index 23f6501..73a1760 100644 --- a/test/codefuzz_by_caller_test.c +++ b/test/codefuzz_by_caller_test.c @@ -7,6 +7,8 @@ #include #include #include +#include +#include #include "../include/map2check.h" @@ -16,7 +18,26 @@ * codegen-units = 1, so linking any symbol from it pulls in * LLVMFuzzerTestOneInput's unresolved reference to __map2check_main__ too — * see the same requirement documented in tests/ffi_harness.c). */ +/* Set by main for the iterations whose program rejects the input. */ +static int reject_input = 0; +static int ran_past_assume = 0; +static int completed = 0; + +/* An exit(0) in place of the unwind would also end with status 0. */ +static void require_completion(void) { + if (!completed) { + _exit(97); + } +} + int __map2check_main__(void) { + if (reject_input) { + /* A failed assume abandons this input and unwinds out of this C + * frame back to LLVMFuzzerTestOneInput (develop: nondet_cancel). */ + __VERIFIER_assume(__VERIFIER_nondet_int() == 42); + ran_past_assume++; + return 0; + } (void)__VERIFIER_nondet_bool(); (void)__VERIFIER_nondet_char(); (void)__VERIFIER_nondet_uchar(); @@ -30,10 +51,12 @@ int __map2check_main__(void) { (void)__VERIFIER_nondet_double(); (void)__VERIFIER_nondet_size_t(); (void)__VERIFIER_nondet_pointer(); + (void)__VERIFIER_nondet_loff_t(); + (void)__VERIFIER_nondet_sector_t(); + free(__VERIFIER_nondet_pchar()); - /* Only truthy conditions here: a falsy one calls exit(0)/abort() by - * design (see rust/src/nondet/libfuzzer.rs), which would end this - * process instead of just this check. */ + /* Truthy conditions here; the falsy assume runs in the iterations of + * main below. A falsy assert aborts by design. */ __VERIFIER_assume(1); __VERIFIER_assert(1); map2check_fuzzer_assume(1); @@ -52,5 +75,21 @@ int main(void) { rc = LLVMFuzzerTestOneInput(second_input, sizeof(second_input)); assert(rc == 0); + /* Input "A": develop's reader rewinds, so x == 0x41414141, not 0, and + * the assume rejects it; the four bytes of 42 pass. */ + atexit(require_completion); + reject_input = 1; + const uint8_t seed_a[] = {'A'}; + const uint8_t forty_two[] = {0x2a, 0x00, 0x00, 0x00}; + for (int i = 0; i < 10; i++) { + rc = LLVMFuzzerTestOneInput(seed_a, sizeof(seed_a)); + assert(rc == 0); + } + assert(ran_past_assume == 0); + rc = LLVMFuzzerTestOneInput(forty_two, sizeof(forty_two)); + assert(rc == 0); + assert(ran_past_assume == 1); + completed = 1; + return rc; } diff --git a/tests/ffi_harness.c b/tests/ffi_harness.c index 6c7d10c..1563994 100644 --- a/tests/ffi_harness.c +++ b/tests/ffi_harness.c @@ -31,7 +31,28 @@ * (rust/src/nondet/libfuzzer.rs only defines this under cfg(test); the real * static lib expects the linking program — here, this harness — to provide * it). */ +/* With `harness_reject_inputs` set, it plays a program whose assume + * rejects every input whose first int is not 7 (develop: nondet_cancel), so + * the unwind of a failed __VERIFIER_assume crosses this C frame. */ +static int harness_reject_inputs = 0; +static int harness_after_assume = 0; +static int harness_completed = 0; + +static void harness_require_completion(void) { + if (!harness_completed) { + fprintf(stderr, "ffi_harness: process ended before main returned\n"); + _exit(97); + } +} + int __map2check_main__(void) { + if (harness_reject_inputs) { + char *s = __VERIFIER_nondet_pchar(); + int x = __VERIFIER_nondet_int(); + free(s); + __VERIFIER_assume(x == 7); + harness_after_assume++; + } return 0; } @@ -47,6 +68,17 @@ static void run_pre_init_boundary(void) { char char_value = 'x'; float float_value = 0.5f; double double_value = 0.5; + unsigned int unsigned_value = 1; + unsigned long ulong_value = 1; + unsigned char uchar_value = 1; + short short_value = 1; + unsigned short ushort_value = 1; + int bool_value = 1; + size_t size_t_value = 1; + long long loff_t_value = 1; + unsigned long sector_t_value = 1; + void *pointer_value = NULL; + char *pchar_value = NULL; int stack_obj = 0; void *heap_obj = malloc(8); if (heap_obj == NULL) { @@ -86,6 +118,19 @@ static void run_pre_init_boundary(void) { (void)map2check_save_nondet_log_char(1, 0, CHAR_ID, &char_value, "main"); (void)map2check_save_nondet_log_float(1, 0, FLOAT_ID, &float_value, "main"); (void)map2check_save_nondet_log_double(1, 0, DOUBLE_ID, &double_value, "main"); + (void)map2check_save_nondet_log_unsigned(1, 0, 5, &unsigned_value, "main"); + (void)map2check_save_nondet_log_ulong(1, 0, 6, &ulong_value, "main"); + (void)map2check_save_nondet_log_uchar(1, 0, 8, &uchar_value, "main"); + (void)map2check_save_nondet_log_short(1, 0, 14, &short_value, "main"); + (void)map2check_save_nondet_log_ushort(1, 0, 3, &ushort_value, "main"); + (void)map2check_save_nondet_log_bool(1, 0, 7, &bool_value, "main"); + (void)map2check_save_nondet_log_size_t(1, 0, 10, &size_t_value, "main"); + (void)map2check_save_nondet_log_loff_t(1, 0, 11, &loff_t_value, "main"); + (void)map2check_save_nondet_log_sector_t(1, 0, 12, §or_t_value, "main"); + (void)map2check_save_nondet_log_pointer(1, 0, 2, &pointer_value, "main"); + (void)map2check_save_nondet_log_pchar(1, 0, 9, &pchar_value, "main"); + (void)map2check_save_nondet_log_unsigned(1, 0, 5, NULL, "main"); + (void)map2check_save_nondet_log_pchar(1, 0, 9, NULL, "main"); map2check_is_valid_assert(1, "main", 1); map2check_binop_add_int(1, 1, 1, 0, "main"); @@ -119,6 +164,9 @@ static void run_pre_init_boundary(void) { (void)__VERIFIER_nondet_double(); (void)__VERIFIER_nondet_size_t(); (void)__VERIFIER_nondet_pointer(); + (void)__VERIFIER_nondet_loff_t(); + (void)__VERIFIER_nondet_sector_t(); + free(__VERIFIER_nondet_pchar()); /* truthy conditions only: falsy ones terminate the process by design */ __VERIFIER_assume(1); @@ -145,6 +193,17 @@ static void run_bad_inputs(void) { char char_value = 0; float float_value = 0.0f; double double_value = 0.0; + unsigned int unsigned_value = 1; + unsigned long ulong_value = 1; + unsigned char uchar_value = 1; + short short_value = 1; + unsigned short ushort_value = 1; + int bool_value = 1; + size_t size_t_value = 1; + long long loff_t_value = 1; + unsigned long sector_t_value = 1; + void *pointer_value = NULL; + char *pchar_value = NULL; /* result/state management, pre-init */ map2check_reset(); @@ -192,6 +251,19 @@ static void run_bad_inputs(void) { (void)map2check_save_nondet_log_char(9, 0, CHAR_ID, &char_value, NULL); (void)map2check_save_nondet_log_float(9, 0, FLOAT_ID, &float_value, NULL); (void)map2check_save_nondet_log_double(9, 0, DOUBLE_ID, &double_value, NULL); + (void)map2check_save_nondet_log_unsigned(1, 0, 5, &unsigned_value, NULL); + (void)map2check_save_nondet_log_ulong(1, 0, 6, &ulong_value, NULL); + (void)map2check_save_nondet_log_uchar(1, 0, 8, &uchar_value, NULL); + (void)map2check_save_nondet_log_short(1, 0, 14, &short_value, NULL); + (void)map2check_save_nondet_log_ushort(1, 0, 3, &ushort_value, NULL); + (void)map2check_save_nondet_log_bool(1, 0, 7, &bool_value, NULL); + (void)map2check_save_nondet_log_size_t(1, 0, 10, &size_t_value, NULL); + (void)map2check_save_nondet_log_loff_t(1, 0, 11, &loff_t_value, NULL); + (void)map2check_save_nondet_log_sector_t(1, 0, 12, §or_t_value, NULL); + (void)map2check_save_nondet_log_pointer(1, 0, 2, &pointer_value, NULL); + (void)map2check_save_nondet_log_pchar(1, 0, 9, &pchar_value, NULL); + (void)map2check_save_nondet_log_unsigned(1, 0, 5, NULL, NULL); + (void)map2check_save_nondet_log_pchar(1, 0, 9, NULL, NULL); /* assert and overflow family with null function name / extreme values */ map2check_is_valid_assert(10, NULL, 0); @@ -229,6 +301,9 @@ static void run_bad_inputs(void) { (void)__VERIFIER_nondet_double(); (void)__VERIFIER_nondet_size_t(); (void)__VERIFIER_nondet_pointer(); + (void)__VERIFIER_nondet_loff_t(); + (void)__VERIFIER_nondet_sector_t(); + free(__VERIFIER_nondet_pchar()); /* __VERIFIER_assume/assert/map2check_fuzzer_assume only with a truthy * condition here: a falsy one calls exit(0)/abort() by design (see @@ -315,6 +390,9 @@ static void run_valid_pass(void) { (void)__VERIFIER_nondet_double(); (void)__VERIFIER_nondet_size_t(); (void)__VERIFIER_nondet_pointer(); + (void)__VERIFIER_nondet_loff_t(); + (void)__VERIFIER_nondet_sector_t(); + free(__VERIFIER_nondet_pchar()); /* __VERIFIER_assume/assert/map2check_fuzzer_assume only with a truthy * condition here: a falsy one calls exit(0)/abort() by design (see @@ -627,6 +705,42 @@ int main(void) { } } + /* A failed __VERIFIER_assume abandons only the current input: the + * unwind crosses __map2check_main__ (a C frame), LLVMFuzzerTestOneInput + * returns 0, and the next input runs (develop: nondet_cancel). */ + /* An exit(0) instead of the unwind would also end with status 0, so + * from here on an early exit is turned into a failure. Registered after + * the forking scenarios above, whose children exit on purpose. */ + atexit(harness_require_completion); + { + const uint8_t rejected[] = {0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00}; + const uint8_t accepted[] = {0x00, 0x00, 0x00, 0x00, 0x07, 0x00, 0x00, 0x00}; + const uint8_t wrapped[] = {0x07}; + harness_reject_inputs = 1; + for (int i = 0; i < 100; i++) { + if (LLVMFuzzerTestOneInput(rejected, sizeof(rejected)) != 0) { + return 1; + } + } + if (harness_after_assume != 0) { + fprintf(stderr, "assume did not abandon the input\n"); + return 1; + } + if (LLVMFuzzerTestOneInput(accepted, sizeof(accepted)) != 0 || + harness_after_assume != 1) { + fprintf(stderr, "accepted input did not run to the end\n"); + return 1; + } + /* {0x07}: pchar length 0x07070707 (capped, 4095 chars from the + * wrap), then x reads 07 07 07 07: rejected. */ + if (LLVMFuzzerTestOneInput(wrapped, sizeof(wrapped)) != 0 || + harness_after_assume != 1) { + return 1; + } + harness_reject_inputs = 0; + } + + harness_completed = 1; map2check_success(); return 0; } diff --git a/tests/legacy_golden/README.md b/tests/legacy_golden/README.md index e17e3b2..aa078bd 100644 --- a/tests/legacy_golden/README.md +++ b/tests/legacy_golden/README.md @@ -32,9 +32,10 @@ compiled with `clang-16`. Each binary runs in a fresh directory. | `POSIX-MEMALIGN` | `AnalysisModeMemtrack.c` | memory prelude, `q` on the stack, `q = BLOCK2` written into its slot, `map2check_posix(&q, 32)`, `map2check_success` (both blocks leak) | | `ASSERT` | `AnalysisModeAssert.c` | `map2check_assert(0, ...)` | | `TARGET-REACHED` | `AnalysisModeNone.c` (`REACHABILITY_MODE`) | `map2check_target_function` | +| `NONDET-TYPES` | `AnalysisModeAssert.c` | one `map2check_nondet_` of each of the 16 `NONDET_TYPE`s, with the value as `NonDetPass.cpp` casts it to `int`, then `map2check_assert(0, ...)` | `drivers/golden.h` holds the shared calls: three basic blocks (one line -repeated), five nondet values (int, uint, char, long, double) and the +repeated), five nondet values (int, uint, char, long, double; the char zero-extended, as `NonDetPass.cpp` passes it) and the memory prelude (`p` on the stack, `&main`, `p = malloc(16)`). Two accommodations make the output reproducible and complete: From 6214215b280a2e0aa08c87c32885c899c9d1b7b4 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 23:21:28 -0400 Subject: [PATCH 32/63] docs: refresh the symbol matrix after the nondet exports Only the three wasm rows (Task 2.8) are still missing from include/map2check.h. --- docs/cutover/symbol-matrix.tsv | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/cutover/symbol-matrix.tsv b/docs/cutover/symbol-matrix.tsv index f474836..b279924 100644 --- a/docs/cutover/symbol-matrix.tsv +++ b/docs/cutover/symbol-matrix.tsv @@ -22,37 +22,37 @@ map2check_init map2check_set_mode+map2check_init 1 map2check_load map2check_check_load 1 map2check_malloc map2check_map_malloc 1 map2check_non_det_bool __VERIFIER_nondet_bool 1 -map2check_nondet_bool map2check_save_nondet_log_bool 0 +map2check_nondet_bool map2check_save_nondet_log_bool 1 map2check_non_det_char __VERIFIER_nondet_char 1 map2check_nondet_char map2check_save_nondet_log_char 1 map2check_non_det_double __VERIFIER_nondet_double 1 map2check_nondet_double map2check_save_nondet_log_double 1 map2check_non_det_int __VERIFIER_nondet_int 1 map2check_nondet_int map2check_save_nondet_log_int 1 -map2check_non_det_loff_t __VERIFIER_nondet_loff_t 0 -map2check_nondet_loff_t map2check_save_nondet_log_loff_t 0 +map2check_non_det_loff_t __VERIFIER_nondet_loff_t 1 +map2check_nondet_loff_t map2check_save_nondet_log_loff_t 1 map2check_non_det_long __VERIFIER_nondet_long 1 map2check_nondet_long map2check_save_nondet_log_long 1 -map2check_non_det_pchar __VERIFIER_nondet_pchar 0 -map2check_nondet_pchar map2check_save_nondet_log_pchar 0 +map2check_non_det_pchar __VERIFIER_nondet_pchar 1 +map2check_nondet_pchar map2check_save_nondet_log_pchar 1 map2check_non_det_pointer __VERIFIER_nondet_pointer 1 -map2check_nondet_pointer map2check_save_nondet_log_pointer 0 -map2check_non_det_sector_t __VERIFIER_nondet_sector_t 0 -map2check_nondet_sector_t map2check_save_nondet_log_sector_t 0 +map2check_nondet_pointer map2check_save_nondet_log_pointer 1 +map2check_non_det_sector_t __VERIFIER_nondet_sector_t 1 +map2check_nondet_sector_t map2check_save_nondet_log_sector_t 1 map2check_non_det_short __VERIFIER_nondet_short 1 -map2check_nondet_short map2check_save_nondet_log_short 0 +map2check_nondet_short map2check_save_nondet_log_short 1 map2check_non_det_size_t __VERIFIER_nondet_size_t 1 -map2check_nondet_size_t map2check_save_nondet_log_size_t 0 +map2check_nondet_size_t map2check_save_nondet_log_size_t 1 map2check_non_det_uchar __VERIFIER_nondet_uchar 1 -map2check_nondet_uchar map2check_save_nondet_log_uchar 0 +map2check_nondet_uchar map2check_save_nondet_log_uchar 1 map2check_non_det_uint __VERIFIER_nondet_uint 1 map2check_nondet_uint map2check_save_nondet_log_uint 1 map2check_non_det_ulong __VERIFIER_nondet_ulong 1 -map2check_nondet_ulong map2check_save_nondet_log_ulong 0 +map2check_nondet_ulong map2check_save_nondet_log_ulong 1 map2check_non_det_unsigned __VERIFIER_nondet_uint 1 -map2check_nondet_unsigned map2check_save_nondet_log_unsigned 0 +map2check_nondet_unsigned map2check_save_nondet_log_unsigned 1 map2check_non_det_ushort __VERIFIER_nondet_ushort 1 -map2check_nondet_ushort map2check_save_nondet_log_ushort 0 +map2check_nondet_ushort map2check_save_nondet_log_ushort 1 map2check_non_static_alloca map2check_map_non_static_alloca 1 map2check_posix map2check_map_posix_memalign 1 map2check_success map2check_success 1 From 8b9da9f0ae679a657a43ad0629753c53d374cf04 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 23:47:09 -0400 Subject: [PATCH 33/63] docs: point the nondet comments at develop's actual source lines --- docs/api-deltas.md | 26 +++++++++++----------- docs/semantic-deltas.md | 6 ++--- rust/src/ffi.rs | 2 +- rust/src/nondet/libfuzzer.rs | 16 ++++++------- rust/src/output/legacy_files.rs | 2 +- tests/legacy_golden/drivers/NONDET-TYPES.c | 2 +- 6 files changed, 27 insertions(+), 27 deletions(-) diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 5057498..d331854 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -63,17 +63,17 @@ build with `--features libfuzzer`. | `map2check_save_nondet_log_float` | `non_det_log_t *map2check_save_nondet_log_float(int line, int scope, enum var_type_t type_var, float *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_double` | `non_det_log_t *map2check_save_nondet_log_double(int line, int scope, enum var_type_t type_var, double *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_char` | `non_det_log_t *map2check_save_nondet_log_char(int line, int scope, enum var_type_t type_var, char *value, const char *function_name)` | returns `non_det_log_t *`; `char` value | same | `char` is `i8` on the Rust side; see “`char` ABI”. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_unsigned` | `non_det_log_t *map2check_save_nondet_log_unsigned(int line, int scope, int type_var, unsigned int *value, const char *function_name)` | — | new | Develop `map2check_nondet_unsigned` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UNSIGNED (5), value printed with `%u`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_ulong` | `non_det_log_t *map2check_save_nondet_log_ulong(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_ulong` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type ULONG (6), low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_uchar` | `non_det_log_t *map2check_save_nondet_log_uchar(int line, int scope, int type_var, unsigned char *value, const char *function_name)` | — | new | Develop `map2check_nondet_uchar` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UCHAR (8), value zero-extended; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_short` | `non_det_log_t *map2check_save_nondet_log_short(int line, int scope, int type_var, short *value, const char *function_name)` | — | new | Develop `map2check_nondet_short` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SHORT (14), value zero-extended as the pass does (`-2` is logged as `65534`); see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_ushort` | `non_det_log_t *map2check_save_nondet_log_ushort(int line, int scope, int type_var, unsigned short *value, const char *function_name)` | — | new | Develop `map2check_nondet_ushort` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type USHORT (3), value zero-extended; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_bool` | `non_det_log_t *map2check_save_nondet_log_bool(int line, int scope, int type_var, int *value, const char *function_name)` | — | new | Develop `map2check_nondet_bool` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type BOOL (7); develop's `bool` is `typedef int bool` (`NonDetGenerator.h:74`), so the pass widens the `_Bool` to `int` before storing it; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_size_t` | `non_det_log_t *map2check_save_nondet_log_size_t(int line, int scope, int type_var, size_t *value, const char *function_name)` | — | new | Develop `map2check_nondet_size_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SIZE_T (10), low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_loff_t` | `non_det_log_t *map2check_save_nondet_log_loff_t(int line, int scope, int type_var, long long *value, const char *function_name)` | — | new | Develop `map2check_nondet_loff_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type LOFF_T (11); `loff_t` is glibc's `long long`; low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_sector_t` | `non_det_log_t *map2check_save_nondet_log_sector_t(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_sector_t` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SECTOR_T (12); `sector_t` is `unsigned long` (`NonDetGenerator.h:112`); low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_pointer` | `non_det_log_t *map2check_save_nondet_log_pointer(int line, int scope, int type_var, void **value, const char *function_name)` | — | new | Develop `map2check_nondet_pointer` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type POINTER (2); the address, logged as its low 32 bits (the pass's `ptrtoint` to `i32`); see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_pchar` | `non_det_log_t *map2check_save_nondet_log_pchar(int line, int scope, int type_var, char **value, const char *function_name)` | — | new | Develop `map2check_nondet_pchar` (`NonDetLog.c:96-130`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type PCHAR (9); the string's address, logged as its low 32 bits; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_unsigned` | `non_det_log_t *map2check_save_nondet_log_unsigned(int line, int scope, int type_var, unsigned int *value, const char *function_name)` | — | new | Develop `map2check_nondet_unsigned` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UNSIGNED (5), value printed with `%u`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_ulong` | `non_det_log_t *map2check_save_nondet_log_ulong(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_ulong` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type ULONG (6), low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_uchar` | `non_det_log_t *map2check_save_nondet_log_uchar(int line, int scope, int type_var, unsigned char *value, const char *function_name)` | — | new | Develop `map2check_nondet_uchar` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UCHAR (8), value zero-extended; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_short` | `non_det_log_t *map2check_save_nondet_log_short(int line, int scope, int type_var, short *value, const char *function_name)` | — | new | Develop `map2check_nondet_short` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SHORT (14), value zero-extended as the pass does (`-2` is logged as `65534`); see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_ushort` | `non_det_log_t *map2check_save_nondet_log_ushort(int line, int scope, int type_var, unsigned short *value, const char *function_name)` | — | new | Develop `map2check_nondet_ushort` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type USHORT (3), value zero-extended; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_bool` | `non_det_log_t *map2check_save_nondet_log_bool(int line, int scope, int type_var, int *value, const char *function_name)` | — | new | Develop `map2check_nondet_bool` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type BOOL (7); develop's `bool` is `typedef int bool` (`NonDetGenerator.h:74`), so the pass widens the `_Bool` to `int` before storing it; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_size_t` | `non_det_log_t *map2check_save_nondet_log_size_t(int line, int scope, int type_var, size_t *value, const char *function_name)` | — | new | Develop `map2check_nondet_size_t` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SIZE_T (10), low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_loff_t` | `non_det_log_t *map2check_save_nondet_log_loff_t(int line, int scope, int type_var, long long *value, const char *function_name)` | — | new | Develop `map2check_nondet_loff_t` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type LOFF_T (11); `loff_t` is glibc's `long long`; low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_sector_t` | `non_det_log_t *map2check_save_nondet_log_sector_t(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_sector_t` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SECTOR_T (12); `sector_t` is `unsigned long` (`NonDetGenerator.h:112`); low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_pointer` | `non_det_log_t *map2check_save_nondet_log_pointer(int line, int scope, int type_var, void **value, const char *function_name)` | — | new | Develop `map2check_nondet_pointer` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type POINTER (2); the address, logged as its low 32 bits (the pass's `ptrtoint` to `i32`); see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_pchar` | `non_det_log_t *map2check_save_nondet_log_pchar(int line, int scope, int type_var, char **value, const char *function_name)` | — | new | Develop `map2check_nondet_pchar` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type PCHAR (9); the string's address, logged as its low 32 bits; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | | `map2check_save_basic_block_log` | `bbtrack_log_t *map2check_save_basic_block_log(int32_t line, const char *function_name)` | returns `bbtrack_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_is_in_trackbb_container` | `int32_t map2check_is_in_trackbb_container(int32_t line)` | `map2check_is_in_trackbb_container` | same | | | `map2check_map_alloca` | `void map2check_map_alloca(const char *var_name, const void *ptr_address, int32_t size, int32_t size_primitive, int32_t line_number, int32_t scope)` | `container_memtracklog.h:41` | same | | @@ -123,7 +123,7 @@ build with `--features libfuzzer`. | `__VERIFIER_nondet_pointer` | `void *__VERIFIER_nondet_pointer(void)` | — | new | Feature `libfuzzer`; replaces `map2check_gen_data_fuzzer_*`. Reads `sizeof` bytes like develop's AFL++ generator, rewinding at the end of the input (SD-18). | | `__VERIFIER_nondet_loff_t` | `int64_t __VERIFIER_nondet_loff_t(void)` | `map2check_gen_data_fuzzer_loff_t` | new | Feature `libfuzzer`. Core target of the pass's `map2check_non_det_loff_t` (#6); `loff_t` is glibc's `long long`, eight bytes (SD-18). | | `__VERIFIER_nondet_sector_t` | `unsigned long __VERIFIER_nondet_sector_t(void)` | `map2check_gen_data_fuzzer_sector_t` | new | Feature `libfuzzer`. Core target of `map2check_non_det_sector_t` (#6); develop's `sector_t` is `unsigned long` (`NonDetGenerator.h:112`), eight bytes (SD-18). | -| `__VERIFIER_nondet_pchar` | `char *__VERIFIER_nondet_pchar(void)` | `map2check_gen_data_fuzzer_pchar` | new | Feature `libfuzzer`. Core target of `map2check_non_det_pchar` (#6). An `unsigned` length, NULL for 0, capped at 4096, then `length - 1` chars and a NUL, as `NonDetGeneratorAFL.c:101-117`; the string comes from the C `malloc` and the program may `free` it (SD-18). | +| `__VERIFIER_nondet_pchar` | `char *__VERIFIER_nondet_pchar(void)` | `map2check_gen_data_fuzzer_pchar` | new | Feature `libfuzzer`. Core target of `map2check_non_det_pchar` (#6). An `unsigned` length, NULL for 0, capped at 4096, then `length - 1` chars and a NUL, as `NonDetGeneratorAFL.c:101-118`; the string comes from the C `malloc` and the program may `free` it (SD-18). | | `__VERIFIER_assume` | `void __VERIFIER_assume(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assume(0)` abandons the current fuzzer input and `LLVMFuzzerTestOneInput` returns 0, as develop's `nondet_cancel`; with no iteration running it calls `exit(0)` (SD-07). Defined `extern "C-unwind"`; see “`__VERIFIER_assume` under a fuzzer”. | | `map2check_fuzzer_assume` | `void map2check_fuzzer_assume(int32_t condition)` | `map2check_fuzzer_assume(int)` (`nondet_gen_libfuzzer.h:41`) | changed | Feature `libfuzzer`. Same behaviour as `__VERIFIER_assume` (SD-07). | | `__VERIFIER_error` | `void __VERIFIER_error(void)` | — | new | Feature `libfuzzer`. Records REACHABILITY, prints the JSON report when the mode is unset, then aborts (SD-08). | @@ -297,7 +297,7 @@ one row of each type. `type_var` is only kept in the returned record. `__VERIFIER_assume(0)` and `map2check_fuzzer_assume(0)` abandon the current input, as develop's `nondet_cancel` does with `longjmp` -(`NonDetGeneratorAFL.c:31-37`, `:184-196`), and the fuzzer continues with +(`NonDetGeneratorAFL.c:31-37`, `:171-184`), and the fuzzer continues with the next input in the same process (SD-07). The mechanism is an unwind, not `longjmp`. Rust cannot call `setjmp` diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 422d1ea..0b1c990 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -252,7 +252,7 @@ running (a program linked without a fuzzer driver), or in a **Legacy (Map2Check `develop`):** `nondet_assume` calls `nondet_cancel`, which `longjmp`s back to the `setjmp` in the `__AFL_LOOP` of `main`, and the -loop moves on to the next input (`NonDetGeneratorAFL.c:31-37`, `:184-196`). +loop moves on to the next input (`NonDetGeneratorAFL.c:31-37`, `:171-184`). The Rust library does the same with an unwind instead of `longjmp`: Rust cannot call `setjmp`, and a `longjmp` over Rust frames is undefined when one of them has a destructor. Outside the `__AFL_LOOP` develop has no @@ -577,10 +577,10 @@ map2check_binop_add_signed(INT64_MAX, 1, 64, 1, "main"); develop's `NonDetGeneratorAFL.c` does: `sizeof(type)` bytes per value, in order, copied in memory order (little-endian on x86-64; `:67-99`), and past the end of the input the reader rewinds, returning `data[0]` once without -advancing (`get_next_input_from_afl`, `:56-63`). With the 1-byte seed `A` +advancing (`get_next_input_from_afl`, `:55-62`). With the 1-byte seed `A` an `int` is `0x41414141`, as in develop. `pchar` reads an `unsigned` length, returns NULL for 0, caps it at 4096 and reads `length - 1` chars -(`:101-117`); the string comes from the C `malloc`. The +(`:101-118`); the string comes from the C `malloc`. The `map2check_save_nondet_log_` exports write the `klee_log.csv` row develop writes for the same program: the type number of `enum NONDET_TYPE` and the value as `NonDetPass.cpp` hands it to the C wrapper (zero-extended diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index ea26e1e..651f49f 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -488,7 +488,7 @@ macro_rules! nondet_ffi { }; } -// One export per `map2check_nondet_` of develop (`NonDetLog.c:96-130`), +// One export per `map2check_nondet_` of develop (`NonDetLog.c:105-150`), // each taking a pointer to the value in the C type `NonDetGenerator.h` gives // that family. The `type_var` argument is only kept in the returned record: // the `enum NONDET_TYPE` of the row comes from the export, as in develop. diff --git a/rust/src/nondet/libfuzzer.rs b/rust/src/nondet/libfuzzer.rs index 552f6ca..56c8048 100644 --- a/rust/src/nondet/libfuzzer.rs +++ b/rust/src/nondet/libfuzzer.rs @@ -13,12 +13,12 @@ // generators, :76-99): little-endian on x86-64, the layout KLEE writes // into a `.ktest` for `klee_make_symbolic(&v, sizeof v)`; // - past the end of the input the reader rewinds (`get_next_input_from_afl`, -// :56-63): the read that finds the index at the end returns `data[0]` +// :55-62): the read that finds the index at the end returns `data[0]` // without advancing, so after every wrap `data[0]` is read twice; // - `pchar` reads an `unsigned` length, returns NULL for 0, caps it at 4096, -// and fills `length - 1` chars plus a NUL (:101-117); +// and fills `length - 1` chars plus a NUL (:101-118); // - a failed `__VERIFIER_assume` abandons the current input and the loop -// moves on to the next one (`nondet_cancel`/`setjmp`, :31-37, :184-196). +// moves on to the next one (`nondet_cancel`/`setjmp`, :31-37, :171-184). use std::sync::atomic::{AtomicBool, AtomicPtr, AtomicUsize, Ordering}; @@ -210,7 +210,7 @@ pub extern "C" fn __VERIFIER_nondet_pointer() -> *mut std::ffi::c_void { const MAX_FUZZED_STRING: u32 = 4096; /// A NUL-terminated string from the input, allocated with the C `malloc` so -/// the program can `free` it (`NonDetGeneratorAFL.c:103-117`): an +/// the program can `free` it (`NonDetGeneratorAFL.c:103-118`): an /// `unsigned` length, NULL when it is 0 (or when `malloc` fails), capped at /// 4096, then `length - 1` chars, each read like `__VERIFIER_nondet_char`. #[no_mangle] @@ -246,7 +246,7 @@ pub extern "C" fn __VERIFIER_nondet_pchar() -> *mut std::os::raw::c_char { // // `__VERIFIER_assume(0)` inside a fuzzer iteration abandons that input, as // develop's `nondet_assume` → `nondet_cancel` → `longjmp` back to the -// `__AFL_LOOP` does (NonDetGeneratorAFL.c:31-37, :184-196): no exit, no +// `__AFL_LOOP` does (NonDetGeneratorAFL.c:31-37, :171-184): no exit, no // abort, and the next input runs in the same process. // // Mechanism (design note): a `longjmp` from Rust back to a `setjmp` would @@ -407,7 +407,7 @@ mod tests { // memory order, into the value (`get_bytes_from_afl`, :67-74; the // generators, :76-99), so a value is the little-endian reading of its // bytes on x86-64. Past the end of the input `get_next_input_from_afl` - // (:56-63) rewinds: the read that finds the index at the end returns + // (:55-62) rewinds: the read that finds the index at the end returns // data[0] WITHOUT advancing, so data[0] is read twice after every wrap. /// develop's `get_next_input_from_afl`, transcribed: the bytes `count` @@ -429,7 +429,7 @@ mod tests { #[test] fn develop_stream_model_rewinds_and_repeats_the_first_byte() { - // Hand-checked against NonDetGeneratorAFL.c:56-63. + // Hand-checked against NonDetGeneratorAFL.c:55-62. assert_eq!(develop_stream(&[1, 2], 7), [1, 2, 1, 1, 2, 1, 1]); assert_eq!(develop_stream(&[0x41], 4), [0x41; 4]); } @@ -578,7 +578,7 @@ mod tests { assert!(__VERIFIER_nondet_pchar().is_null()); } - // ── pchar (NonDetGeneratorAFL.c:101-117) ───────────────────────────────── + // ── pchar (NonDetGeneratorAFL.c:101-118) ───────────────────────────────── extern "C" { fn free(ptr: *mut std::ffi::c_void); diff --git a/rust/src/output/legacy_files.rs b/rust/src/output/legacy_files.rs index 43467ba..1131efd 100644 --- a/rust/src/output/legacy_files.rs +++ b/rust/src/output/legacy_files.rs @@ -173,7 +173,7 @@ pub fn write_trackbb_log(dir: &Path, rows: &[BasicBlockEntry]) -> io::Result<()> /// (`Map2CheckTypes.h:41-58`) of a nondet row, as develop writes them. /// /// `NonDetPass.cpp` hands each value to its `map2check_nondet_` wrapper as -/// a C `int` (`NonDetLog.c:96-130`): `int`, `unsigned`, `uint` as they are +/// a C `int` (`NonDetLog.c:105-150`): `int`, `unsigned`, `uint` as they are /// (`NONDET_IMPL_HELPER`, `:260-280`); the other integer kinds through /// `CreateIntegerCast(v, i32, isSigned = false)` (`:240-258`, `:281-290`), /// which zero-extends the narrow ones (`char` -3 arrives as 253) and diff --git a/tests/legacy_golden/drivers/NONDET-TYPES.c b/tests/legacy_golden/drivers/NONDET-TYPES.c index 0040843..3ed65c9 100644 --- a/tests/legacy_golden/drivers/NONDET-TYPES.c +++ b/tests/legacy_golden/drivers/NONDET-TYPES.c @@ -2,7 +2,7 @@ * Assert mode: one nondet value of every NONDET_TYPE (Map2CheckTypes.h:41-58), * then a failed assert, so the witness writes klee_log.csv. * - * Each value reaches its map2check_nondet_ wrapper (NonDetLog.c:96-130) + * Each value reaches its map2check_nondet_ wrapper (NonDetLog.c:105-150) * as NonDetPass.cpp passes it: * - int, unsigned, uint, double: the call result itself (NONDET_IMPL_HELPER, * :260-275, used at :277-280); From 8881adcb3f996acab614885e8c91510f6ba2dee5 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Thu, 15 Oct 2026 23:47:09 -0400 Subject: [PATCH 34/63] docs: record the D1 re-measurement; conditions 3 and 4 not met (Task 2.5b) Five sequential 30 s afl-fuzz campaigns per variant, --cpus 4, B2 built from develop's sources. Medians, B1/B2: prog42 119% exec/s; the assume-heavy program 55% (70% with CmpLog), because every rejected input unwinds the stack where develop longjmps. Time to first crash with CmpLog on prog42: 822 ms against 190 ms (4.33x). Fixing the short-input reader did not close it (749 ms before): the uninstrumented Rust reader leaves the corpus at the 1-byte seed, while develop's instrumented reader adds 4-byte entries for input-to-state. With seed "AAAA" both find it by op:its in about 175 ms. The D1 route is unchanged. --- docs/cutover/decisoes.md | 4 ++ docs/cutover/spike-llvm16.md | 111 +++++++++++++++++++++++++++++++++++ 2 files changed, 115 insertions(+) diff --git a/docs/cutover/decisoes.md b/docs/cutover/decisoes.md index 3135056..dbd697f 100644 --- a/docs/cutover/decisoes.md +++ b/docs/cutover/decisoes.md @@ -124,11 +124,15 @@ A aprovação aceita a proposta junto com estas condições. - B1 e B2 rodaram ao mesmo tempo, na mesma máquina (`--cpus 4`). A remedição exige a saída em arquivo do Rust (Task 2.4), um `__VERIFIER_assume` que não encerre o processo persistente e programas com muitos `assume`. O Rust faz `exit(0)` (`libfuzzer.rs:197-206`); o C faz `longjmp` e segue para a próxima entrada. As campanhas devem rodar em sequência, com as mesmas flags e pelo menos 5 repetições. Abaixo de 90% do C, o `afl` passa para B2 (port do `__AFL_LOOP`). + + **Status (Task 2.5b, 2026-10-15): NÃO ATENDIDA.** Remedido em sequência, 5 repetições, `--cpus 4`, B2 compilado das fontes de `develop` e gravando arquivos, `assume` que abandona a entrada sem encerrar o processo (`spike-llvm16.md`, “Remedição (Task 2.5b)”). Medianas de exec/s, B1/B2: `prog42` 116 390 / 98 060 (**119%**) sem CmpLog e 115 510 / 97 434 (119%) com CmpLog; programa com `assume` 64 018 / 115 493 (**55%**) sem CmpLog e 75 394 / 107 069 (**70%**) com CmpLog. A perda vem do desenrolar de pilha que rejeita cada entrada (o B2 usa `longjmp`). A rota não foi alterada; a decisão fica com o controlador. 4. **O tempo até o primeiro crash com CmpLog é critério de aceitação da D1.** Ele será remedido no corpus no gate 4.2. O frontend sempre roda o afl-fuzz com `-c `, então para guardas de valor mágico o que conta é o tempo até o crash, não exec/s. No spike, com CmpLog, o B1 levou cerca de 100× mais execuções que o B2 (280 900 contra 2 748), e o B2 achou o crash pela etapa de input-to-state (`op:its`). **Hipótese principal,** tirada do código e não medida (o spike não foi refeito): a diferença está em entradas mais curtas que o valor lido. O `read_bytes` do Rust (`libfuzzer.rs:54-71`) devolve `None` se faltarem bytes, e `read_u32` vira **0**. O C (`get_next_input_from_afl`) **volta ao início do buffer**. Com a semente `A` (1 byte), o C lê `x = 0x41414141`, que aparece na entrada. O CmpLog vê `0x41414141 == 42`, acha os bytes e os troca por `2a 00 00 00`. No Rust, `x = 0` não vem de nenhum byte da entrada, e o input-to-state não tem o que substituir. Só a mutação havoc, depois de crescer a entrada para 4 bytes, chega a `x = 42`. Hipótese secundária, do revisor: a leitura passa por código Rust não instrumentado. A correção provável da principal é a Task 2.5 alinhar a semântica de entrada curta à do C. Isso também é exigido pela condição 9. + + **Status (Task 2.5b, 2026-10-15): NÃO ATENDIDA.** Com o leitor já igual ao do C, mediana de 5 campanhas `prog42` com CmpLog: B1 **822 ms** (86 511 execs, sempre `op:havoc`), B2 **190 ms** (544 execs, sempre `op:its`), ou 4,33× (o limite é 2×). Antes da correção do leitor (`0d5b1f6`), o B1 tinha mediana de 749 ms: a hipótese principal não explica a diferença. Vale a secundária: o runtime C instrumentado gera arestas (o ramo de volta ao início do buffer) que põem entradas de 4 bytes no corpus, onde o input-to-state troca o operando; o leitor Rust não é instrumentado e o corpus do B1 fica só com a semente `A`, de 1 byte. Com a semente `AAAA`, B1 e B2 acham o crash por `op:its` em cerca de 175 ms. No programa com `assume`, com CmpLog, a condição é atendida (B1 145 ms, B2 298 ms). A rota não foi alterada; a decisão fica com o controlador. 5. **MSRV 1.72 no que o build `symex` compila,** com os custos (a)–(c) acima e o `clear_poison` sob `#[cfg(not(feature = "klee"))]`. Responsável: Task 2.7, com o job de CI em 1.72.1. 6. **LTO fat e `--strip-debug` do runtime `symex` são obrigatórios.** O `-g` do programa analisado é preservado. Responsável: Task 2.7. 7. **Concretizar antes de serializar no `symex`.** O `serde_json` formatando valores simbólicos levou o `prog.c` de 2 para 22 caminhos. O gerador KLEE do Rust precisa concretizar (`klee_get_value_*`) o que vai para o log e para o relatório, como o C faz ao escrever por chamada externa. Responsável: Task 2.7. diff --git a/docs/cutover/spike-llvm16.md b/docs/cutover/spike-llvm16.md index ec8d7d3..ffa3d6e 100644 --- a/docs/cutover/spike-llvm16.md +++ b/docs/cutover/spike-llvm16.md @@ -395,6 +395,117 @@ KLEE: releases no GitHub, consultadas em 2026-10-15 com `curl -s https://api.git Resposta: **sim, há KLEE com suporte a LLVM 19 (3.3)**, mas a release saiu hoje e não foi testada aqui. Não verifiquei se o AFL++ 4.40c compila com LLVM 19. Estimativa da Rota C: os passes são P–M (troca mecânica mais revisão do tratamento de debug info, cerca de 2 a 4 dias). Somam-se a reconstrução da imagem (clang/LLVM 19, KLEE 3.3 + STP/Z3 + klee-uclibc para LLVM 19, AFL++ recompilado) e a revalidação dos baselines de corpus, porque todo o pipeline muda de versão. No total, G (1 a 2 semanas), com o risco de ser a primeira versão de KLEE com suporte completo a LLVM 19. +## Remedição (Task 2.5b) + +Data: 2026-10-15. Biblioteca no commit `f4e4908` (branch `cutover/runtime-rust`): o leitor de entrada segue o `NonDetGeneratorAFL.c` (volta ao início do buffer, `sizeof(tipo)` bytes por valor) e um `__VERIFIER_assume` falso abandona a entrada sem encerrar o processo persistente (desenrola até o `LLVMFuzzerTestOneInput`, que devolve 0). Responde às condições 3 e 4 da D1. + +### Montagem + +Tudo na imagem `map2check-dev:cutover-rust`, com `docker run --rm --cpus 4`. As campanhas rodaram **em sequência**, um fuzzer por vez, 5 repetições de cada variante. + +- **B1:** `libmap2check.a` do perfil `runtime` com `--features libfuzzer` (rustc 1.82 da imagem), ligado com `afl-clang-fast -g -O2 -fsanitize=fuzzer` ao programa com `-Dmain=__map2check_main__`; o binário CmpLog com `AFL_LLVM_CMPLOG=1`. Comandos iguais aos de `b1build.sh` do spike. +- **B2:** o runtime C de `develop` compilado **das fontes** (`modules/backend/library/lib/*.c`, `clang-16 -c -emit-llvm`, como o `CMakeLists.txt` da biblioteca), na ligação de alcançabilidade do `caller.cpp` (`Map2CheckFunctions`, `TrackBBLog`, `NonDetLog`, `PropertyGenerator`, `ContainerRealloc`, `AnalysisModeNone`, `NonDetGeneratorAFL`, `WitnessGenerationNone`), `llvm-link-16` e `afl-clang-fast -g -O2`. Comandos iguais aos de `b2build.sh` do spike. +- **Programas:** `prog42.c` / `prog42_c.c` do spike (o `prog.c` com `abort()` em `x == 42`, para o crash ser contado) e um programa com muitos `assume`, em duas formas como as anteriores: + +```c +/* assume.c (B1); assume_c.c (B2) chama map2check_non_det_int, + * map2check_assume (= nondet_assume, Map2CheckFunctions.c:45) e map2check_init(0) */ +int main(void) { + map2check_init(); + int x = __VERIFIER_nondet_int(); + __VERIFIER_assume(x > 100); + __VERIFIER_assume(x < 200); + if (x == 150) abort(); + map2check_success(); + return 0; +} +``` + +- **Campanha:** semente `A` (a do frontend, `caller.cpp:904-907`) e o ambiente do frontend (`caller.cpp:919-925`) sem `AFL_BENCH_UNTIL_CRASH`, para a campanha durar os 30 s inteiros: + +```bash +AFL_NO_UI=1 AFL_NO_AFFINITY=1 AFL_SKIP_CPUFREQ=1 AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES=1 \ +AFL_CRASHING_SEEDS_AS_NEW_CRASH=1 afl-fuzz -i seed -o runs/--- -V 30 \ + [-c ./-.cmplog.afl] -- ./-.afl +``` + + Ordem: para cada repetição, `prog42` e depois `assume`; sem e com CmpLog; B1 e depois B2. O 1º crash vem do nome do primeiro arquivo em `crashes/` (`time`, `execs`, `op`). + +Antes das campanhas, um teste diferencial ligou um programa que lê um valor de cada tipo (16 tipos, `pchar` incluído) ao B1 e ao B2 e comparou as saídas para entradas de 1, 2, 4, 7, 7 (`pchar` de tamanho 5), 37, 64 e 200 bytes: **idênticas em todas** (B2 lendo da stdin, B1 pelo `LLVMFuzzerTestOneInput`). + +### Dados brutos + +| rep | rota | programa | CmpLog | `execs_per_sec` | `execs_done` | `stability` | 1º crash (`time` ms, `execs`, `op`) | +|---|---|---|---|---|---|---|---| +| 1 | B1 | `prog42` | não | 113373,29 | 3 401 312 | 100,00% | 3636, 397 890, `havoc` | +| 1 | B2 | `prog42` | não | 96080,26 | 2 882 504 | 75,68% | 82, 1 677, `havoc` | +| 1 | B1 | `prog42` | sim | 113733,28 | 3 412 112 | 100,00% | 231, 14 988, `havoc` | +| 1 | B2 | `prog42` | sim | 95805,70 | 2 876 183 | 78,38% | 207, 544, `its` | +| 1 | B1 | `assume` | não | 67556,28 | 2 027 837 | 100,00% | 246, 19 009, `havoc` | +| 1 | B2 | `assume` | não | 115492,63 | 3 465 010 | 80,00% | 295, 27 045, `havoc` | +| 1 | B1 | `assume` | sim | 102674,16 | 3 080 430 | 100,00% | 247, 7 843, `havoc` | +| 1 | B2 | `assume` | sim | 123441,82 | 3 703 378 | 100,00% | 204, 1 534, `havoc` | +| 2 | B1 | `prog42` | não | 116556,51 | 3 496 812 | 100,00% | 201, 15 586, `havoc` | +| 2 | B2 | `prog42` | não | 98477,62 | 2 954 427 | 75,68% | 1436, 130 559, `havoc` | +| 2 | B1 | `prog42` | sim | 115136,56 | 3 454 212 | 100,00% | 822, 86 511, `havoc` | +| 2 | B2 | `prog42` | sim | 97434,11 | 2 924 777 | 78,38% | 189, 538, `its` | +| 2 | B1 | `assume` | não | 59714,79 | 1 792 220 | 100,00% | 109, 2 636, `havoc` | +| 2 | B2 | `assume` | não | 116131,52 | 3 484 178 | 80,00% | 82, 1 306, `havoc` | +| 2 | B1 | `assume` | sim | 59930,52 | 1 803 729 | 100,00% | 145, 8 689, `havoc` | +| 2 | B2 | `assume` | sim | 101517,68 | 3 046 850 | 100,00% | 469, 37 277, `havoc` | +| 3 | B1 | `prog42` | não | 116879,84 | 3 506 512 | 100,00% | 1991, 229 706, `havoc` | +| 3 | B2 | `prog42` | não | 97799,48 | 2 934 180 | 75,68% | 671, 56 521, `havoc` | +| 3 | B1 | `prog42` | sim | 115509,88 | 3 465 412 | 100,00% | 298, 25 229, `havoc` | +| 3 | B2 | `prog42` | sim | 98499,00 | 2 955 167 | 78,38% | 196, 545, `its` | +| 3 | B1 | `assume` | não | 100495,07 | 3 015 053 | 100,00% | 513, 48 445, `havoc` | +| 3 | B2 | `assume` | não | 106180,13 | 3 185 510 | 80,00% | 140, 9 025, `havoc` | +| 3 | B1 | `assume` | sim | 80445,25 | 2 414 001 | 100,00% | 158, 8 272, `havoc` | +| 3 | B2 | `assume` | sim | 107068,98 | 3 212 819 | 100,00% | 298, 13 751, `havoc` | +| 4 | B1 | `prog42` | não | 116096,53 | 3 483 012 | 100,00% | 801, 83 658, `havoc` | +| 4 | B2 | `prog42` | não | 98181,92 | 2 945 654 | 75,68% | 5951, 594 288, `havoc` | +| 4 | B1 | `prog42` | sim | 115666,54 | 3 470 112 | 100,00% | 1346, 146 540, `havoc` | +| 4 | B2 | `prog42` | sim | 96819,37 | 2 904 678 | 78,38% | 190, 539, `its` | +| 4 | B1 | `assume` | não | 64018,25 | 1 922 532 | 100,00% | 208, 16 015, `havoc` | +| 4 | B2 | `assume` | não | 112247,26 | 3 367 530 | 80,00% | 291, 26 376, `havoc` | +| 4 | B1 | `assume` | sim | 75394,05 | 2 261 897 | 100,00% | 103, 1 936, `havoc` | +| 4 | B2 | `assume` | sim | 123763,01 | 3 713 014 | 100,00% | 629, 46 719, `havoc` | +| 5 | B1 | `prog42` | não | 116389,85 | 3 491 812 | 100,00% | 590, 59 409, `havoc` | +| 5 | B2 | `prog42` | não | 98060,26 | 2 942 102 | 75,68% | 416, 34 493, `havoc` | +| 5 | B1 | `prog42` | sim | 115703,21 | 3 471 212 | 100,00% | 836, 88 894, `havoc` | +| 5 | B2 | `prog42` | sim | 97957,20 | 2 938 814 | 78,38% | 185, 650, `its` | +| 5 | B1 | `assume` | não | 57883,57 | 1 736 565 | 100,00% | 103, 3 406, `havoc` | +| 5 | B2 | `assume` | não | 115598,85 | 3 468 081 | 80,00% | 118, 4 449, `havoc` | +| 5 | B1 | `assume` | sim | 64970,70 | 1 949 446 | 100,00% | 91, 1 497, `havoc` | +| 5 | B2 | `assume` | sim | 99577,21 | 2 987 416 | 100,00% | 227, 4 740, `havoc` | + +### Medianas (n = 5) + +| Programa | CmpLog | exec/s B1 | exec/s B2 | B1/B2 | 1º crash B1 (ms; execs) | 1º crash B2 (ms; execs) | B1 ÷ B2 | +|---|---|---|---|---|---|---|---| +| `prog42` | não | 116 390 | 98 060 | **118,7%** | 801; 83 658 | 671; 56 521 | 1,19× | +| `prog42` | sim | 115 510 | 97 434 | **118,6%** | **822; 86 511** | **190; 544** | **4,33×** | +| `assume` | não | 64 018 | 115 493 | **55,4%** | 208; 16 015 | 140; 9 025 | 1,49× | +| `assume` | sim | 75 394 | 107 069 | **70,4%** | 145; 7 843 | 298; 13 751 | 0,49× | + +Todas as 40 campanhas acharam o crash. Estabilidade: B1 100% em todas; B2 75,68% (`prog42`), 78,38% (`prog42` com CmpLog), 80% e 100% (`assume`). + +### Leitura + +**Throughput (condição 3).** No `prog42` o B1 fica em 119% do B2, agora com o B2 gravando arquivos e as campanhas em sequência. No programa com `assume`, quase toda entrada é rejeitada, e o B1 cai para **55%** (sem CmpLog) e **70%** (com CmpLog) do B2. O B2 rejeita com `longjmp`, que custa poucos nanossegundos; o B1 rejeita com um desenrolar de pilha (`resume_unwind` + `catch_unwind`), que passa pelo unwinder (`_Unwind_RaiseException`, busca de FDE) a cada rejeição. O B1 também varia mais entre repetições (57 884 a 100 495 exec/s sem CmpLog), o que é consistente com o custo crescer com a fração de entradas rejeitadas. + +**Tempo até o 1º crash com CmpLog (condição 4).** No `prog42` o B2 achou o crash pela etapa input-to-state (`op:its`) nas 5 repetições, em 185 a 207 ms; o B1, por havoc nas 5, com mediana de 822 ms. Corrigir o leitor (a hipótese principal) **não fechou a diferença**. Linha de base com a biblioteca de antes desta task (`0d5b1f6`, leitor que devolvia 0), mesmo protocolo, só `prog42` com CmpLog: 102, 749, 2 845, 115 e 1 083 ms, mediana **749 ms**, todas por havoc. A diferença está no corpus. No B2, o runtime C é instrumentado, e o ramo de volta ao início do buffer de `get_next_input_from_afl` é uma aresta nova: entradas de 2 e 4 bytes (`41 41`, `c0 c0 c0 c0`) entram no corpus em 50 execuções (`corpus_count` 4), e o input-to-state troca os 4 bytes do `c0c0c0c0` por `2a 00 00 00`. No B1 o leitor é Rust não instrumentado, o corpus fica só com a semente `A` (`corpus_count` 1), e o input-to-state não consegue pôr um operando de 4 bytes numa entrada de 1 byte. É a hipótese secundária da condição 4. + +Verificação (informativa, não altera o protocolo): a mesma campanha `prog42` com CmpLog, mas com a semente `AAAA` e `AFL_BENCH_UNTIL_CRASH=1`, 5 repetições sequenciais. O B1 achou o crash em 171 a 184 ms (13 execs, `op:its`) e o B2 em 170 a 184 ms (25 execs, `op:its`): as duas rotas ficam iguais quando a semente já tem o tamanho do valor lido. + +No programa com `assume`, com CmpLog, o B1 achou o crash antes do B2 (mediana 145 ms contra 298 ms). + +### Veredito contra os critérios da D1 + +- **Condição 3 (throughput ≥ 90% do B2): NÃO ATENDIDA.** Atendida no `prog42` (119%), não atendida no programa com `assume` (55% sem CmpLog, 70% com CmpLog). +- **Condição 4 (1º crash com CmpLog não pior que 2× a mediana do B2): NÃO ATENDIDA** no `prog42` (822 ms contra 190 ms, 4,33×); atendida no programa com `assume` (0,49×). + +A rota da D1 não foi alterada. Caminhos possíveis, para decisão do controlador: (a) para a condição 4, uma semente no frontend (Task 3.x) com pelo menos o tamanho do maior valor lido (8 bytes cobrem `long`, `double` e ponteiros); com 4 bytes e um `int`, o teste acima igualou B1 e B2; (b) para a condição 3, rejeitar a entrada sem desenrolar a pilha, por exemplo com um `setjmp` num trecho C ligado à biblioteca (exige compilar C no `build.rs`, e o `longjmp` só é definido se nenhum frame Rust pulado tiver destrutor); (c) a alternativa nomeada na D1, o B2 (port do `__AFL_LOOP`). + ## Como reproduzir Na branch `spike/llvm16`: `spikes/llvm16/{b1build.sh,b2build.sh,fuzz.sh}`, `klee-rt/`, `klee-rt-std/`, `full172/` (com o `Cargo.lock` v3 usado) e `Dockerfile.rust`. From c1b5be5168fffccdbbabe262bb9ac86e49a33641 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 09:20:16 -0400 Subject: [PATCH 35/63] chore: commit the D1 re-measurement scripts and raw results (Task 2.5b) scripts/cutover/d1-remeasure/ holds the docker scripts, the B1/B2 programs, the differential type programs and the raw TSVs of the re-measurement, plus corpus_count and the queue of every campaign and the -O2 unwind-flag matrix of the harness. The README gives the mounts and commands. --- scripts/cutover/d1-remeasure/README.md | 41 +++++++++++++ scripts/cutover/d1-remeasure/assume.c | 16 +++++ scripts/cutover/d1-remeasure/assume_c.c | 18 ++++++ scripts/cutover/d1-remeasure/bench.sh | 31 ++++++++++ scripts/cutover/d1-remeasure/build-routes.sh | 38 ++++++++++++ scripts/cutover/d1-remeasure/build_old.sh | 19 ++++++ scripts/cutover/d1-remeasure/diff.sh | 16 +++++ scripts/cutover/d1-remeasure/flags16.sh | 18 ++++++ scripts/cutover/d1-remeasure/prog42.c | 13 ++++ scripts/cutover/d1-remeasure/prog42_c.c | 14 +++++ .../d1-remeasure/results/corpus-v1.tsv | 41 +++++++++++++ .../d1-remeasure/results/differential.txt | 8 +++ .../cutover/d1-remeasure/results/flags16.txt | 13 ++++ .../d1-remeasure/results/results-old.tsv | 6 ++ .../d1-remeasure/results/results-seed4.tsv | 10 ++++ .../d1-remeasure/results/results-v1.tsv | 41 +++++++++++++ scripts/cutover/d1-remeasure/seed4.sh | 11 ++++ scripts/cutover/d1-remeasure/types_b1.c | 59 +++++++++++++++++++ scripts/cutover/d1-remeasure/types_b2.c | 46 +++++++++++++++ 19 files changed, 459 insertions(+) create mode 100644 scripts/cutover/d1-remeasure/README.md create mode 100644 scripts/cutover/d1-remeasure/assume.c create mode 100644 scripts/cutover/d1-remeasure/assume_c.c create mode 100755 scripts/cutover/d1-remeasure/bench.sh create mode 100755 scripts/cutover/d1-remeasure/build-routes.sh create mode 100755 scripts/cutover/d1-remeasure/build_old.sh create mode 100755 scripts/cutover/d1-remeasure/diff.sh create mode 100755 scripts/cutover/d1-remeasure/flags16.sh create mode 100644 scripts/cutover/d1-remeasure/prog42.c create mode 100644 scripts/cutover/d1-remeasure/prog42_c.c create mode 100644 scripts/cutover/d1-remeasure/results/corpus-v1.tsv create mode 100644 scripts/cutover/d1-remeasure/results/differential.txt create mode 100644 scripts/cutover/d1-remeasure/results/flags16.txt create mode 100644 scripts/cutover/d1-remeasure/results/results-old.tsv create mode 100644 scripts/cutover/d1-remeasure/results/results-seed4.tsv create mode 100644 scripts/cutover/d1-remeasure/results/results-v1.tsv create mode 100755 scripts/cutover/d1-remeasure/seed4.sh create mode 100644 scripts/cutover/d1-remeasure/types_b1.c create mode 100644 scripts/cutover/d1-remeasure/types_b2.c diff --git a/scripts/cutover/d1-remeasure/README.md b/scripts/cutover/d1-remeasure/README.md new file mode 100644 index 0000000..ecbb474 --- /dev/null +++ b/scripts/cutover/d1-remeasure/README.md @@ -0,0 +1,41 @@ +# D1 re-measurement (Task 2.5b) + +Scripts, programs and raw results of the re-measurement of D1 conditions 3 +and 4 (`docs/cutover/decisoes.md`). The write-up is +`docs/cutover/spike-llvm16.md`, section “Remedição (Task 2.5b)”. + +Every script runs inside `map2check-dev:cutover-rust` (built from the spike's +`Dockerfile.rust`) with these mounts: + +| Mount | What | +|---|---| +| `/src` | this repository, read-write (`build.rs` rewrites `include/map2check.h`) | +| `/runtime` | `modules/backend/library` of the Map2Check `develop` checkout, read-only | +| `/bench` | a **copy** of this directory in a scratch location (the scripts write their builds and campaign outputs there) | +| `/old` | only for `build_old.sh`: a `git worktree` of `0d5b1f6` | + +```sh +cp -r scripts/cutover/d1-remeasure "$SCRATCH/bench" +run() { docker run --rm --cpus 4 -v "$PWD":/src \ + -v "$MAP2CHECK/modules/backend/library":/runtime:ro \ + -v "$SCRATCH/bench":/bench map2check-dev:cutover-rust bash "$@"; } +run /bench/build-routes.sh v1 # B1/B2 binaries and the type programs, in build-v1/ +run /bench/diff.sh # differential: 16 types, B1 vs B2 (creates inputs/ and out/) +run /bench/bench.sh build-v1 5 # 40 sequential 30 s campaigns -> stdout TSV +run /bench/seed4.sh # informative: seed "AAAA", stop at first crash +run /bench/flags16.sh # -O2 harness per unwind flag (clang-16, afl-clang-fast) +``` + +`build_old.sh` needs the extra `-v "$WORKTREE":/old` mount. + +| File | Role | +|---|---| +| `prog42.c`, `prog42_c.c` | the spike's program for B1 and as the passes leave it for B2 | +| `assume.c`, `assume_c.c` | the assume-heavy program, same two forms | +| `types_b1.c`, `types_b2.c` | read one value of each nondet type and print it | +| `results/results-v1.tsv` | raw stats of the 40 campaigns | +| `results/corpus-v1.tsv` | `corpus_count` and queue entries of the same 40 campaigns | +| `results/results-old.tsv` | baseline with the library before Task 2.5b | +| `results/results-seed4.tsv` | informative runs with the 4-byte seed | +| `results/differential.txt` | output of `diff.sh` | +| `results/flags16.txt` | output of `flags16.sh` (fix round 1) | diff --git a/scripts/cutover/d1-remeasure/assume.c b/scripts/cutover/d1-remeasure/assume.c new file mode 100644 index 0000000..4642ce6 --- /dev/null +++ b/scripts/cutover/d1-remeasure/assume.c @@ -0,0 +1,16 @@ +/* Assume-heavy program for the D1 re-measurement (Task 2.5b): most inputs + * are rejected by an assume, and the bug sits behind a magic value. */ +#include +extern void map2check_init(void); +extern void map2check_success(void); +extern int __VERIFIER_nondet_int(void); +extern void __VERIFIER_assume(int); +int main(void) { + map2check_init(); + int x = __VERIFIER_nondet_int(); + __VERIFIER_assume(x > 100); + __VERIFIER_assume(x < 200); + if (x == 150) abort(); + map2check_success(); + return 0; +} diff --git a/scripts/cutover/d1-remeasure/assume_c.c b/scripts/cutover/d1-remeasure/assume_c.c new file mode 100644 index 0000000..8105fef --- /dev/null +++ b/scripts/cutover/d1-remeasure/assume_c.c @@ -0,0 +1,18 @@ +/* B2 reference: assume.c as the Map2Check passes leave it for the C runtime + * (main renamed, nondet and assume calls renamed, map2check_init(0)). The + * C runtime's map2check_assume is nondet_assume (Map2CheckFunctions.c). */ +#include +extern void map2check_init(int isSvComp); +extern void map2check_success(void); +extern int map2check_non_det_int(void); +extern void map2check_assume(int); +int __map2check_main__(int argc, char **argv) { + (void)argc; (void)argv; + map2check_init(0); + int x = map2check_non_det_int(); + map2check_assume(x > 100); + map2check_assume(x < 200); + if (x == 150) abort(); + map2check_success(); + return 0; +} diff --git a/scripts/cutover/d1-remeasure/bench.sh b/scripts/cutover/d1-remeasure/bench.sh new file mode 100755 index 0000000..639eadf --- /dev/null +++ b/scripts/cutover/d1-remeasure/bench.sh @@ -0,0 +1,31 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0-only +# Task 2.5b D1 re-measurement: sequential 30 s afl-fuzz campaigns, never two +# fuzzers at once. usage: bench.sh +# Same environment as the frontend (caller.cpp:919-925) minus +# AFL_BENCH_UNTIL_CRASH (the campaign must run its whole 30 s for exec/s); +# seed "A" as the frontend's placeholder (caller.cpp:904-907). +B=/bench/$1; REPS=$2 +export AFL_NO_UI=1 AFL_NO_AFFINITY=1 AFL_SKIP_CPUFREQ=1 AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES=1 AFL_CRASHING_SEEDS_AS_NEW_CRASH=1 +cd $B +mkdir -p seed && printf A > seed/seed +mkdir -p runs +echo -e "rep\tvariant\tprog\tcmplog\texecs_per_sec\texecs_done\tsaved_crashes\tstability\tfirst_crash" +for rep in $(seq 1 $REPS); do + for prog in prog42 assume; do + for cmp in no yes; do + for v in b1 b2; do + out=runs/$v-$prog-$cmp-$rep + C=""; [ $cmp = yes ] && C="-c ./$v-$prog.cmplog.afl" + afl-fuzz -i seed -o $out -V 30 $C -- ./$v-$prog.afl > $out.log 2>&1 + s=$out/default/fuzzer_stats + first=$(ls $out/default/crashes 2>/dev/null | grep '^id:' | sort | head -1) + printf "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n" $rep $v $prog $cmp \ + "$(awk -F': ' '/^execs_per_sec/{print $2}' $s)" \ + "$(awk -F': ' '/^execs_done/{print $2}' $s)" \ + "$(awk -F': ' '/^saved_crashes/{print $2}' $s)" \ + "$(awk -F': ' '/^stability/{print $2}' $s)" "${first:--}" + done + done + done +done diff --git a/scripts/cutover/d1-remeasure/build-routes.sh b/scripts/cutover/d1-remeasure/build-routes.sh new file mode 100755 index 0000000..3454ca9 --- /dev/null +++ b/scripts/cutover/d1-remeasure/build-routes.sh @@ -0,0 +1,38 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0-only +# Builds B1 (Rust lib + AFL++ libFuzzer driver) and B2 (develop's C runtime, +# NonDetGeneratorAFL.c) for the Task 2.5b measurements, plus the +# differential type programs. Mounts: /src repo (rw: build.rs writes the +# header), /runtime develop's modules/backend/library (ro), /bench this dir. +set -euo pipefail +export CARGO_TARGET_DIR=/bench/target-docker RUSTUP_TOOLCHAIN=1.82.0 +cd /src && cargo build --profile runtime --features libfuzzer 2>&1 | tail -1 +L=$CARGO_TARGET_DIR/runtime/libmap2check.a +B=/bench/build-$1 +mkdir "$B" && cd "$B" +# develop's runtime as bitcode, as modules/backend/library/lib/CMakeLists.txt +# compiles it (clang -c -emit-llvm), the reachability link of caller.cpp +for f in Map2CheckFunctions TrackBBLog NonDetLog PropertyGenerator ContainerRealloc \ + AnalysisModeNone NonDetGeneratorAFL WitnessGenerationNone; do + clang-16 -c -emit-llvm -I/runtime/header /runtime/lib/$f.c -o $f.bc 2>/dev/null +done +RT="Map2CheckFunctions.bc TrackBBLog.bc NonDetLog.bc PropertyGenerator.bc ContainerRealloc.bc AnalysisModeNone.bc NonDetGeneratorAFL.bc WitnessGenerationNone.bc" +for p in prog42 assume; do + # B1: main renamed by the pass, linked with the native staticlib + clang-16 -emit-llvm -c -Dmain=__map2check_main__ /bench/$p.c -o ${p}_m.bc + afl-clang-fast -g -O2 -fsanitize=fuzzer ${p}_m.bc $L -ldl -lpthread -lm -o b1-$p.afl 2>/dev/null + AFL_LLVM_CMPLOG=1 afl-clang-fast -g -O2 -fsanitize=fuzzer ${p}_m.bc $L -ldl -lpthread -lm -o b1-$p.cmplog.afl 2>/dev/null + # B2: as the passes leave it for the C runtime + clang-16 -emit-llvm -c /bench/${p}_c.c -o ${p}_c.bc + llvm-link-16 ${p}_c.bc $RT -o $p-result.bc + afl-clang-fast -g -O2 $p-result.bc -o b2-$p.afl 2>/dev/null + AFL_LLVM_CMPLOG=1 afl-clang-fast -g -O2 $p-result.bc -o b2-$p.cmplog.afl 2>/dev/null +done +# differential type programs +clang-16 -O2 -I/src/include /bench/types_b1.c $L -ldl -lpthread -lm -o types_b1 +clang-16 -emit-llvm -c /bench/types_b2.c -o types_b2.bc +llvm-link-16 types_b2.bc $RT -o types-result.bc +afl-clang-fast -O2 types-result.bc -o types_b2 2>/dev/null +ls -la b1-* b2-* types_b* +nm b1-prog42.afl | grep -cE " T (LLVMFuzzerTestOneInput|__map2check_main__|__afl_persistent_loop)$" +strings b1-prog42.afl | grep -c SIG_AFL_PERSISTENT diff --git a/scripts/cutover/d1-remeasure/build_old.sh b/scripts/cutover/d1-remeasure/build_old.sh new file mode 100755 index 0000000..a545092 --- /dev/null +++ b/scripts/cutover/d1-remeasure/build_old.sh @@ -0,0 +1,19 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0-only +# Baseline: the library before Task 2.5b (0d5b1f6, a git worktree mounted +# at /old), B1 prog42 with CmpLog, 5 sequential 30 s campaigns. +set -euo pipefail +export CARGO_TARGET_DIR=/bench/target-old RUSTUP_TOOLCHAIN=1.82.0 +cd /old && cargo build --profile runtime --features libfuzzer 2>&1 | tail -1 +L=$CARGO_TARGET_DIR/runtime/libmap2check.a +mkdir /bench/build-old && cd /bench/build-old +clang-16 -emit-llvm -c -Dmain=__map2check_main__ /bench/prog42.c -o prog42_m.bc +afl-clang-fast -g -O2 -fsanitize=fuzzer prog42_m.bc $L -ldl -lpthread -lm -o b0-prog42.afl 2>/dev/null +AFL_LLVM_CMPLOG=1 afl-clang-fast -g -O2 -fsanitize=fuzzer prog42_m.bc $L -ldl -lpthread -lm -o b0-prog42.cmplog.afl 2>/dev/null +export AFL_NO_UI=1 AFL_NO_AFFINITY=1 AFL_SKIP_CPUFREQ=1 AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES=1 AFL_CRASHING_SEEDS_AS_NEW_CRASH=1 +mkdir seed && printf A > seed/seed +for rep in 1 2 3 4 5; do + out=out-$rep + afl-fuzz -i seed -o $out -V 30 -c ./b0-prog42.cmplog.afl -- ./b0-prog42.afl > $out.log 2>&1 + printf "%s\tb0\tprog42\tyes\t%s\t%s\n" $rep "$(awk -F': ' '/^execs_per_sec/{print $2}' $out/default/fuzzer_stats)" "$(ls $out/default/crashes | grep '^id:' | head -1)" +done diff --git a/scripts/cutover/d1-remeasure/diff.sh b/scripts/cutover/d1-remeasure/diff.sh new file mode 100755 index 0000000..385eda9 --- /dev/null +++ b/scripts/cutover/d1-remeasure/diff.sh @@ -0,0 +1,16 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0-only +# Runs both type programs on every input; prints the diff (empty = same). +cd /bench +mkdir -p inputs out +if [ ! -e inputs/one ]; then + printf A > inputs/one; printf '\x01\x02' > inputs/two + printf '\x2a\x00\x00\x00' > inputs/four; printf '\x01\x02\x03\x04\x05\x06\x07' > inputs/seven + printf '\x05\x00\x00\x00hey' > inputs/pchar5 + for n in 37 64 200; do head -c $n /dev/urandom > inputs/rand$n; done +fi +for f in inputs/*; do + ./build-v1/types_b2 < $f > out/$(basename $f).b2 + ./build-v1/types_b1 $f > out/$(basename $f).b1 + if cmp -s out/$(basename $f).b1 out/$(basename $f).b2; then echo "$(basename $f) ($(stat -c %s $f) bytes): identical"; else echo "$(basename $f): DIFFER"; diff out/$(basename $f).b1 out/$(basename $f).b2; fi +done diff --git a/scripts/cutover/d1-remeasure/flags16.sh b/scripts/cutover/d1-remeasure/flags16.sh new file mode 100755 index 0000000..50ede54 --- /dev/null +++ b/scripts/cutover/d1-remeasure/flags16.sh @@ -0,0 +1,18 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0-only +# clang-16 / afl-clang-fast (the pipeline's compilers): attributes of the +# harness's __map2check_main__ and the -O2 harness run per flag set. +export CARGO_TARGET_DIR=/bench/target-docker RUSTUP_TOOLCHAIN=1.82.0 +cd /src && cargo build --release --all-features 2>&1 | tail -1 +L=$CARGO_TARGET_DIR/release/libmap2check.a +mkdir -p /bench/flags16 && cd /bench/flags16 +for cc in clang-16 afl-clang-fast; do + for fl in "" "-fexceptions" "-funwind-tables" "-fno-asynchronous-unwind-tables -fno-unwind-tables" "-fexceptions -fno-asynchronous-unwind-tables -fno-unwind-tables"; do + clang-16 -O2 -S -emit-llvm $fl /src/tests/ffi_harness.c -o h.ll 2>/dev/null + a=$(grep -E "^define .*@__map2check_main__" h.ll | grep -o '#[0-9]*' | tail -1) + attrs=$(grep "^attributes $a " h.ll | grep -oE 'nounwind|uwtable(=[a-z]+)?' | tr '\n' ' ') + $cc -O2 $fl /src/tests/ffi_harness.c $L -ldl -lpthread -lm -o h 2>/dev/null || { echo "$cc [$fl] build failed"; continue; } + ./h >/dev/null 2>h.err; rc=$? + echo "$cc [${fl:-default}] main attrs: ${attrs}| rc=$rc $(grep -m1 -E 'failed to initiate|ended before' h.err)" + done +done diff --git a/scripts/cutover/d1-remeasure/prog42.c b/scripts/cutover/d1-remeasure/prog42.c new file mode 100644 index 0000000..5278b2b --- /dev/null +++ b/scripts/cutover/d1-remeasure/prog42.c @@ -0,0 +1,13 @@ +/* Variant of prog.c for the fuzzers: x == 42 aborts, so reaching it is a + * crash afl-fuzz can count. Same calls otherwise. */ +#include +extern void map2check_init(void); +extern void map2check_success(void); +extern int __VERIFIER_nondet_int(void); +int main(void) { + map2check_init(); + int x = __VERIFIER_nondet_int(); + if (x == 42) abort(); + map2check_success(); + return 0; +} diff --git a/scripts/cutover/d1-remeasure/prog42_c.c b/scripts/cutover/d1-remeasure/prog42_c.c new file mode 100644 index 0000000..cd149ed --- /dev/null +++ b/scripts/cutover/d1-remeasure/prog42_c.c @@ -0,0 +1,14 @@ +/* B2 reference: prog42.c as the Map2Check passes would leave it for the C + * runtime (main renamed, nondet call renamed to the generator symbol). */ +#include +extern void map2check_init(int isSvComp); +extern void map2check_success(void); +extern int map2check_non_det_int(void); +int __map2check_main__(int argc, char **argv) { + (void)argc; (void)argv; + map2check_init(0); + int x = map2check_non_det_int(); + if (x == 42) abort(); + map2check_success(); + return 0; +} diff --git a/scripts/cutover/d1-remeasure/results/corpus-v1.tsv b/scripts/cutover/d1-remeasure/results/corpus-v1.tsv new file mode 100644 index 0000000..b8c9f24 --- /dev/null +++ b/scripts/cutover/d1-remeasure/results/corpus-v1.tsv @@ -0,0 +1,41 @@ +variant prog cmplog rep corpus_count queue (name: first 16 bytes hex) +b1 prog42 no 1 1 id:000000:41 +b1 prog42 no 2 1 id:000000:41 +b1 prog42 no 3 1 id:000000:41 +b1 prog42 no 4 1 id:000000:41 +b1 prog42 no 5 1 id:000000:41 +b2 prog42 no 1 4 id:000000:41 id:000001:b8b8b8b8 id:000002:414141 id:000003:49 +b2 prog42 no 2 4 id:000000:41 id:000001:c2c2c2c2 id:000002:4141 id:000003:9e41 +b2 prog42 no 3 4 id:000000:41 id:000001:3333 id:000002:41414141 id:000003:0f +b2 prog42 no 4 4 id:000000:41 id:000001:4545 id:000002:429d9d9d id:000003:0040 +b2 prog42 no 5 4 id:000000:41 id:000001:4141 id:000002:48484848 id:000003:ffe5ffffff483b48 +b1 prog42 yes 1 1 id:000000:41 +b1 prog42 yes 2 1 id:000000:41 +b1 prog42 yes 3 1 id:000000:41 +b1 prog42 yes 4 1 id:000000:41 +b1 prog42 yes 5 1 id:000000:41 +b2 prog42 yes 1 4 id:000000:41 id:000001:4141 id:000002:c0c0c0c0 id:000003:0004 +b2 prog42 yes 2 4 id:000000:41 id:000001:4141 id:000002:ffffffff id:000003:be41 +b2 prog42 yes 3 4 id:000000:41 id:000001:5f41 id:000002:7a7a7a7a id:000003:444444387b8000004444444444444444 +b2 prog42 yes 4 4 id:000000:41 id:000001:4141 id:000002:41414141 id:000003:41 +b2 prog42 yes 5 4 id:000000:41 id:000001:7c7c7c7c id:000002:4100 id:000003:6f +b1 assume no 1 3 id:000000:41 id:000001:e3 id:000002:7f000000 +b1 assume no 2 3 id:000000:41 id:000001:ecececec id:000002:7f000000 +b1 assume no 3 3 id:000000:41 id:000001:cf id:000002:6e000000 +b1 assume no 4 3 id:000000:41 id:000001:c1 id:000002:80000000 +b1 assume no 5 3 id:000000:41 id:000001:be id:000002:7f000000 +b2 assume no 1 6 id:000000:41 id:000001:89 id:000002:1101 id:000003:e1e1e141 id:000004:7f000000c7 id:000005:4e00f68ae8 +b2 assume no 2 6 id:000000:41 id:000001:41414141 id:000002:4141 id:000003:af id:000004:7f00000020 id:000005:0000 +b2 assume no 3 6 id:000000:41 id:000001:90 id:000002:41414141 id:000003:4141 id:000004:7f000000 id:000005:41414131 +b2 assume no 4 6 id:000000:41 id:000001:9f id:000002:d2d2 id:000003:41ececec id:000004:7f0000001d1d1d1d1d1d1d1d1d1d1d1d id:000005:d2d0d2 +b2 assume no 5 6 id:000000:41 id:000001:c1 id:000002:4141 id:000003:68686868 id:000004:7e000000 id:000005:4141 +b1 assume yes 1 3 id:000000:41 id:000001:d0 id:000002:6d000000 +b1 assume yes 2 3 id:000000:41 id:000001:ba id:000002:7f000000 +b1 assume yes 3 3 id:000000:41 id:000001:d2 id:000002:7f000000 +b1 assume yes 4 3 id:000000:41 id:000001:ef0017 id:000002:80000000 +b1 assume yes 5 3 id:000000:41 id:000001:a0e0 id:000002:7f000000 +b2 assume yes 1 6 id:000000:41 id:000001:d1 id:000002:9f9bee71 id:000003:8a40 id:000004:7f000000 id:000005:dc597f +b2 assume yes 2 6 id:000000:41 id:000001:7f7f7f7f id:000002:be id:000003:4141 id:000004:80000000 id:000005:52525252 +b2 assume yes 3 6 id:000000:41 id:000001:9d id:000002:4141 id:000003:f4f4f4f4 id:000004:80000000 id:000005:9c +b2 assume yes 4 6 id:000000:41 id:000001:c6 id:000002:4141 id:000003:99999999 id:000004:b8000000 id:000005:7f7f616f +b2 assume yes 5 6 id:000000:41 id:000001:d8 id:000002:4140 id:000003:41414141 id:000004:80000000 id:000005:41 diff --git a/scripts/cutover/d1-remeasure/results/differential.txt b/scripts/cutover/d1-remeasure/results/differential.txt new file mode 100644 index 0000000..63076fe --- /dev/null +++ b/scripts/cutover/d1-remeasure/results/differential.txt @@ -0,0 +1,8 @@ +four (4 bytes): identical +one (1 bytes): identical +pchar5 (7 bytes): identical +rand200 (200 bytes): identical +rand37 (37 bytes): identical +rand64 (64 bytes): identical +seven (7 bytes): identical +two (2 bytes): identical diff --git a/scripts/cutover/d1-remeasure/results/flags16.txt b/scripts/cutover/d1-remeasure/results/flags16.txt new file mode 100644 index 0000000..42b4b42 --- /dev/null +++ b/scripts/cutover/d1-remeasure/results/flags16.txt @@ -0,0 +1,13 @@ + Finished `release` profile [optimized] target(s) in 16.94s +clang-16 [default] main attrs: nounwind uwtable | rc=0 +clang-16 [-fexceptions] main attrs: uwtable | rc=0 +clang-16 [-funwind-tables] main attrs: nounwind uwtable | rc=0 +/bench/flags16.sh: line 9: 809 Aborted (core dumped) ./h > /dev/null 2> h.err +clang-16 [-fno-asynchronous-unwind-tables -fno-unwind-tables] main attrs: nounwind | rc=134 fatal runtime error: failed to initiate panic, error 5 +clang-16 [-fexceptions -fno-asynchronous-unwind-tables -fno-unwind-tables] main attrs: | rc=0 +afl-clang-fast [default] main attrs: nounwind uwtable | rc=0 +afl-clang-fast [-fexceptions] main attrs: uwtable | rc=0 +afl-clang-fast [-funwind-tables] main attrs: nounwind uwtable | rc=0 +/bench/flags16.sh: line 9: 919 Aborted (core dumped) ./h > /dev/null 2> h.err +afl-clang-fast [-fno-asynchronous-unwind-tables -fno-unwind-tables] main attrs: nounwind | rc=134 fatal runtime error: failed to initiate panic, error 5 +afl-clang-fast [-fexceptions -fno-asynchronous-unwind-tables -fno-unwind-tables] main attrs: | rc=0 diff --git a/scripts/cutover/d1-remeasure/results/results-old.tsv b/scripts/cutover/d1-remeasure/results/results-old.tsv new file mode 100644 index 0000000..59ecde4 --- /dev/null +++ b/scripts/cutover/d1-remeasure/results/results-old.tsv @@ -0,0 +1,6 @@ + Finished `runtime` profile [optimized] target(s) in 16.29s +1 b0 prog42 yes 114823.24 id:000000,sig:06,src:000000,time:102,execs:1267,op:havoc,rep:7 +2 b0 prog42 yes 114983.23 id:000000,sig:06,src:000000,time:749,execs:79386,op:havoc,rep:29 +3 b0 prog42 yes 115866.54 id:000000,sig:06,src:000000,time:2845,execs:337103,op:havoc,rep:53 +4 b0 prog42 yes 116179.86 id:000000,sig:06,src:000000,time:115,execs:4871,op:havoc,rep:8 +5 b0 prog42 yes 116893.17 id:000000,sig:06,src:000000,time:1083,execs:122295,op:havoc,rep:51 diff --git a/scripts/cutover/d1-remeasure/results/results-seed4.tsv b/scripts/cutover/d1-remeasure/results/results-seed4.tsv new file mode 100644 index 0000000..fac57be --- /dev/null +++ b/scripts/cutover/d1-remeasure/results/results-seed4.tsv @@ -0,0 +1,10 @@ +1 b1 id:000000,sig:06,src:000000,time:177,execs:13,op:its,pos:0 +1 b2 id:000000,sig:06,src:000000,time:183,execs:25,op:its,pos:0 +2 b1 id:000000,sig:06,src:000000,time:181,execs:13,op:its,pos:0 +2 b2 id:000000,sig:06,src:000000,time:184,execs:25,op:its,pos:0 +3 b1 id:000000,sig:06,src:000000,time:171,execs:13,op:its,pos:0 +3 b2 id:000000,sig:06,src:000000,time:177,execs:25,op:its,pos:0 +4 b1 id:000000,sig:06,src:000000,time:171,execs:13,op:its,pos:0 +4 b2 id:000000,sig:06,src:000000,time:170,execs:25,op:its,pos:0 +5 b1 id:000000,sig:06,src:000000,time:184,execs:13,op:its,pos:0 +5 b2 id:000000,sig:06,src:000000,time:175,execs:25,op:its,pos:0 diff --git a/scripts/cutover/d1-remeasure/results/results-v1.tsv b/scripts/cutover/d1-remeasure/results/results-v1.tsv new file mode 100644 index 0000000..2a4dd39 --- /dev/null +++ b/scripts/cutover/d1-remeasure/results/results-v1.tsv @@ -0,0 +1,41 @@ +rep variant prog cmplog execs_per_sec execs_done saved_crashes stability first_crash +1 b1 prog42 no 113373.29 3401312 1 100.00% id:000000,sig:06,src:000000,time:3636,execs:397890,op:havoc,rep:48 +1 b2 prog42 no 96080.26 2882504 1 75.68% id:000000,sig:06,src:000002,time:82,execs:1677,op:havoc,rep:14 +1 b1 prog42 yes 113733.28 3412112 1 100.00% id:000000,sig:06,src:000000,time:231,execs:14988,op:havoc,rep:31 +1 b2 prog42 yes 95805.70 2876183 1 78.38% id:000000,sig:06,src:000002,time:207,execs:544,op:its,rep:1 +1 b1 assume no 67556.28 2027837 1 100.00% id:000000,sig:06,src:000002,time:246,execs:19009,op:havoc,rep:1 +1 b2 assume no 115492.63 3465010 1 80.00% id:000000,sig:06,src:000004,time:295,execs:27045,op:havoc,rep:1 +1 b1 assume yes 102674.16 3080430 1 100.00% id:000000,sig:06,src:000002,time:247,execs:7843,op:havoc,rep:1 +1 b2 assume yes 123441.82 3703378 1 100.00% id:000000,sig:06,src:000004,time:204,execs:1534,op:havoc,rep:1 +2 b1 prog42 no 116556.51 3496812 1 100.00% id:000000,sig:06,src:000000,time:201,execs:15586,op:havoc,rep:55 +2 b2 prog42 no 98477.62 2954427 1 75.68% id:000000,sig:06,src:000003,time:1436,execs:130559,op:havoc,rep:14 +2 b1 prog42 yes 115136.56 3454212 1 100.00% id:000000,sig:06,src:000000,time:822,execs:86511,op:havoc,rep:60 +2 b2 prog42 yes 97434.11 2924777 1 78.38% id:000000,sig:06,src:000002,time:189,execs:538,op:its,rep:8 +2 b1 assume no 59714.79 1792220 1 100.00% id:000000,sig:06,src:000002,time:109,execs:2636,op:havoc,rep:2 +2 b2 assume no 116131.52 3484178 1 80.00% id:000000,sig:06,src:000004,time:82,execs:1306,op:havoc,rep:2 +2 b1 assume yes 59930.52 1803729 1 100.00% id:000000,sig:06,src:000002,time:145,execs:8689,op:havoc,rep:26 +2 b2 assume yes 101517.68 3046850 1 100.00% id:000000,sig:06,src:000004,time:469,execs:37277,op:havoc,rep:2 +3 b1 prog42 no 116879.84 3506512 1 100.00% id:000000,sig:06,src:000000,time:1991,execs:229706,op:havoc,rep:2 +3 b2 prog42 no 97799.48 2934180 1 75.68% id:000000,sig:06,src:000002,time:671,execs:56521,op:havoc,rep:16 +3 b1 prog42 yes 115509.88 3465412 1 100.00% id:000000,sig:06,src:000000,time:298,execs:25229,op:havoc,rep:56 +3 b2 prog42 yes 98499.00 2955167 1 78.38% id:000000,sig:06,src:000002,time:196,execs:545,op:its,rep:7 +3 b1 assume no 100495.07 3015053 1 100.00% id:000000,sig:06,src:000002,time:513,execs:48445,op:havoc,rep:3 +3 b2 assume no 106180.13 3185510 1 80.00% id:000000,sig:06,src:000003,time:140,execs:9025,op:havoc,rep:31 +3 b1 assume yes 80445.25 2414001 1 100.00% id:000000,sig:06,src:000002,time:158,execs:8272,op:havoc,rep:2 +3 b2 assume yes 107068.98 3212819 1 100.00% id:000000,sig:06,src:000004,time:298,execs:13751,op:havoc,rep:2 +4 b1 prog42 no 116096.53 3483012 1 100.00% id:000000,sig:06,src:000000,time:801,execs:83658,op:havoc,rep:22 +4 b2 prog42 no 98181.92 2945654 1 75.68% id:000000,sig:06,src:000002,time:5951,execs:594288,op:havoc,rep:15 +4 b1 prog42 yes 115666.54 3470112 1 100.00% id:000000,sig:06,src:000000,time:1346,execs:146540,op:havoc,rep:20 +4 b2 prog42 yes 96819.37 2904678 1 78.38% id:000000,sig:06,src:000002,time:190,execs:539,op:its,rep:14 +4 b1 assume no 64018.25 1922532 1 100.00% id:000000,sig:06,src:000002,time:208,execs:16015,op:havoc,rep:1 +4 b2 assume no 112247.26 3367530 1 80.00% id:000000,sig:06,src:000004,time:291,execs:26376,op:havoc,rep:2 +4 b1 assume yes 75394.05 2261897 1 100.00% id:000000,sig:06,src:000002,time:103,execs:1936,op:havoc,rep:2 +4 b2 assume yes 123763.01 3713014 1 100.00% id:000000,sig:06,src:000004,time:629,execs:46719,op:havoc,rep:1 +5 b1 prog42 no 116389.85 3491812 1 100.00% id:000000,sig:06,src:000000,time:590,execs:59409,op:havoc,rep:16 +5 b2 prog42 no 98060.26 2942102 1 75.68% id:000000,sig:06,src:000001,time:416,execs:34493,op:havoc,rep:56 +5 b1 prog42 yes 115703.21 3471212 1 100.00% id:000000,sig:06,src:000000,time:836,execs:88894,op:havoc,rep:42 +5 b2 prog42 yes 97957.20 2938814 1 78.38% id:000000,sig:06,src:000001,time:185,execs:650,op:its,rep:1 +5 b1 assume no 57883.57 1736565 1 100.00% id:000000,sig:06,src:000002,time:103,execs:3406,op:havoc,rep:1 +5 b2 assume no 115598.85 3468081 1 80.00% id:000000,sig:06,src:000004,time:118,execs:4449,op:havoc,rep:2 +5 b1 assume yes 64970.70 1949446 1 100.00% id:000000,sig:06,src:000002,time:91,execs:1497,op:havoc,rep:8 +5 b2 assume yes 99577.21 2987416 1 100.00% id:000000,sig:06,src:000004,time:227,execs:4740,op:havoc,rep:4 diff --git a/scripts/cutover/d1-remeasure/seed4.sh b/scripts/cutover/d1-remeasure/seed4.sh new file mode 100755 index 0000000..8ae830c --- /dev/null +++ b/scripts/cutover/d1-remeasure/seed4.sh @@ -0,0 +1,11 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0-only +# Informational: prog42 with CmpLog and a 4-byte seed "AAAA", stopping at the first crash. +cd /bench/build-v1 +export AFL_NO_UI=1 AFL_NO_AFFINITY=1 AFL_SKIP_CPUFREQ=1 AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES=1 AFL_CRASHING_SEEDS_AS_NEW_CRASH=1 AFL_BENCH_UNTIL_CRASH=1 +mkdir seed4 && printf AAAA > seed4/seed +for rep in 1 2 3 4 5; do for v in b1 b2; do + out=runs4/$v-$rep; mkdir -p runs4 + afl-fuzz -i seed4 -o $out -V 30 -c ./$v-prog42.cmplog.afl -- ./$v-prog42.afl > $out.log 2>&1 + printf "%s\t%s\t%s\n" $rep $v "$(ls $out/default/crashes | grep '^id:' | head -1)" +done; done diff --git a/scripts/cutover/d1-remeasure/types_b1.c b/scripts/cutover/d1-remeasure/types_b1.c new file mode 100644 index 0000000..cda4068 --- /dev/null +++ b/scripts/cutover/d1-remeasure/types_b1.c @@ -0,0 +1,59 @@ +#include +#include +#include +#include "map2check.h" +#define __map2check_main__ __map2check_main__ +/* Differential check (B1): the Rust library reading one value + * of each type from the input. */ + + + + + + + + + + + + + + + + + + + +int __map2check_main__(void) { int argc = 0; char **argv = NULL; + (void)argc; (void)argv; + printf("char %d\n", __VERIFIER_nondet_char()); + printf("int %d\n", __VERIFIER_nondet_int()); + printf("bool %d\n", __VERIFIER_nondet_bool() != 0); + printf("short %d\n", __VERIFIER_nondet_short()); + printf("long %ld\n", __VERIFIER_nondet_long()); + printf("uchar %u\n", __VERIFIER_nondet_uchar()); + printf("ushort %u\n", __VERIFIER_nondet_ushort()); + printf("uint %u\n", __VERIFIER_nondet_uint()); + printf("unsigned %u\n", __VERIFIER_nondet_uint()); + printf("ulong %lu\n", __VERIFIER_nondet_ulong()); + printf("size_t %zu\n", __VERIFIER_nondet_size_t()); + printf("loff_t %lld\n", (long long)__VERIFIER_nondet_loff_t()); + printf("sector_t %lu\n", __VERIFIER_nondet_sector_t()); + double d = __VERIFIER_nondet_double(); + printf("double %016llx\n", *(unsigned long long *)&d); + printf("pointer %p\n", __VERIFIER_nondet_pointer()); + char *s = __VERIFIER_nondet_pchar(); + printf("pchar %s\n", s ? s : "(null)"); + if (s) { unsigned n = 0; for (char *p = s; *p; p++) n = n * 31 + (unsigned char)*p; printf("pchar_hash %u len %zu\n", n, (size_t)(s ? __builtin_strlen(s) : 0)); } + free(s); + printf("int %d\n", __VERIFIER_nondet_int()); + return 0; +} +/* Feeds the file named on the command line to LLVMFuzzerTestOneInput once. */ +int main(int argc, char **argv) { + FILE *f = fopen(argv[1], "rb"); + static uint8_t buf[1 << 20]; + size_t n = fread(buf, 1, sizeof buf, f); + fclose(f); + return LLVMFuzzerTestOneInput(buf, n); +} diff --git a/scripts/cutover/d1-remeasure/types_b2.c b/scripts/cutover/d1-remeasure/types_b2.c new file mode 100644 index 0000000..f987da4 --- /dev/null +++ b/scripts/cutover/d1-remeasure/types_b2.c @@ -0,0 +1,46 @@ +/* Differential check (B2): develop's NonDetGeneratorAFL.c reading one value + * of each type from the test case on stdin. */ +#include +#include +#include +extern int map2check_non_det_int(void); +extern unsigned map2check_non_det_uint(void); +extern unsigned map2check_non_det_unsigned(void); +extern char map2check_non_det_char(void); +extern unsigned char map2check_non_det_uchar(void); +extern short map2check_non_det_short(void); +extern unsigned short map2check_non_det_ushort(void); +extern long map2check_non_det_long(void); +extern unsigned long map2check_non_det_ulong(void); +extern int map2check_non_det_bool(void); +extern size_t map2check_non_det_size_t(void); +extern loff_t map2check_non_det_loff_t(void); +extern unsigned long map2check_non_det_sector_t(void); +extern double map2check_non_det_double(void); +extern void *map2check_non_det_pointer(void); +extern char *map2check_non_det_pchar(void); +int __map2check_main__(int argc, char **argv) { + (void)argc; (void)argv; + printf("char %d\n", map2check_non_det_char()); + printf("int %d\n", map2check_non_det_int()); + printf("bool %d\n", map2check_non_det_bool() != 0); + printf("short %d\n", map2check_non_det_short()); + printf("long %ld\n", map2check_non_det_long()); + printf("uchar %u\n", map2check_non_det_uchar()); + printf("ushort %u\n", map2check_non_det_ushort()); + printf("uint %u\n", map2check_non_det_uint()); + printf("unsigned %u\n", map2check_non_det_unsigned()); + printf("ulong %lu\n", map2check_non_det_ulong()); + printf("size_t %zu\n", map2check_non_det_size_t()); + printf("loff_t %lld\n", (long long)map2check_non_det_loff_t()); + printf("sector_t %lu\n", map2check_non_det_sector_t()); + double d = map2check_non_det_double(); + printf("double %016llx\n", *(unsigned long long *)&d); + printf("pointer %p\n", map2check_non_det_pointer()); + char *s = map2check_non_det_pchar(); + printf("pchar %s\n", s ? s : "(null)"); + if (s) { unsigned n = 0; for (char *p = s; *p; p++) n = n * 31 + (unsigned char)*p; printf("pchar_hash %u len %zu\n", n, (size_t)(s ? __builtin_strlen(s) : 0)); } + free(s); + printf("int %d\n", map2check_non_det_int()); + return 0; +} From 6d21e7a3cd6f58d3f1bc97b4ab5d2faa41dc66e3 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 09:20:16 -0400 Subject: [PATCH 36/63] ci: run the harness at -O2 and require -fexceptions for the analysed program A failed __VERIFIER_assume unwinds through the program's C frames, which clang marks nounwind unless the program is built with -fexceptions: undefined in LLVM IR, and it only works thanks to the default .eh_frame. Measured at -O2 with clang-16 and afl-clang-fast: default and -funwind-tables pass but stay nounwind; without unwind tables the process aborts ("failed to initiate panic"); -fexceptions drops nounwind and still works when the tables are turned off afterwards. So -fexceptions is the required flag (D1 condition 11, Task 3.2). CI now builds tests/ffi_harness.c at -O2 with and without -fexceptions. Docs: type_var is ignored for the row type and why, the unwind runs __attribute__((cleanup)) handlers that longjmp skips, and the iteration flag is process-wide. --- .github/workflows/CI.yml | 19 +++++++++ docs/api-deltas.md | 78 ++++++++++++++++++++++++++++-------- docs/semantic-deltas.md | 8 +++- rust/src/ffi.rs | 7 +++- rust/src/nondet/libfuzzer.rs | 21 ++++++++-- 5 files changed, 108 insertions(+), 25 deletions(-) diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index 52ea18c..cef1d0d 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -84,6 +84,25 @@ jobs: done [ "$ok" = 1 ] + # The harness rejects inputs with __VERIFIER_assume(0), which unwinds + # through the C frame of its __map2check_main__ (D1 condition 11). + # Optimised builds are where a missing unwind path would show: once + # with -fexceptions, the flag the pipeline must use, and once with + # clang's defaults. + - name: Compilar e executar harness FFI em -O2 (com e sem -fexceptions) + run: | + cargo build --release --all-features + for flags in "-fexceptions" ""; do + clang -O2 $flags -fsanitize=address,undefined -fno-omit-frame-pointer \ + tests/ffi_harness.c target/release/libmap2check.a \ + -ldl -lpthread -lm -o target/ffi_harness_o2 + ok=0 + for i in 1 2 3 4 5; do + if target/ffi_harness_o2; then ok=1; break; fi + done + [ "$ok" = 1 ] || { echo "ffi_harness -O2 $flags falhou"; exit 1; } + done + - name: Compilar e executar test/codeAPI_by_caller_test.c (API núcleo) run: | cargo build --release diff --git a/docs/api-deltas.md b/docs/api-deltas.md index d331854..6116edd 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -63,17 +63,17 @@ build with `--features libfuzzer`. | `map2check_save_nondet_log_float` | `non_det_log_t *map2check_save_nondet_log_float(int line, int scope, enum var_type_t type_var, float *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_double` | `non_det_log_t *map2check_save_nondet_log_double(int line, int scope, enum var_type_t type_var, double *value, const char *function_name)` | returns `non_det_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_save_nondet_log_char` | `non_det_log_t *map2check_save_nondet_log_char(int line, int scope, enum var_type_t type_var, char *value, const char *function_name)` | returns `non_det_log_t *`; `char` value | same | `char` is `i8` on the Rust side; see “`char` ABI”. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_unsigned` | `non_det_log_t *map2check_save_nondet_log_unsigned(int line, int scope, int type_var, unsigned int *value, const char *function_name)` | — | new | Develop `map2check_nondet_unsigned` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UNSIGNED (5), value printed with `%u`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_ulong` | `non_det_log_t *map2check_save_nondet_log_ulong(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_ulong` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type ULONG (6), low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_uchar` | `non_det_log_t *map2check_save_nondet_log_uchar(int line, int scope, int type_var, unsigned char *value, const char *function_name)` | — | new | Develop `map2check_nondet_uchar` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UCHAR (8), value zero-extended; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_short` | `non_det_log_t *map2check_save_nondet_log_short(int line, int scope, int type_var, short *value, const char *function_name)` | — | new | Develop `map2check_nondet_short` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SHORT (14), value zero-extended as the pass does (`-2` is logged as `65534`); see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_ushort` | `non_det_log_t *map2check_save_nondet_log_ushort(int line, int scope, int type_var, unsigned short *value, const char *function_name)` | — | new | Develop `map2check_nondet_ushort` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type USHORT (3), value zero-extended; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_bool` | `non_det_log_t *map2check_save_nondet_log_bool(int line, int scope, int type_var, int *value, const char *function_name)` | — | new | Develop `map2check_nondet_bool` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type BOOL (7); develop's `bool` is `typedef int bool` (`NonDetGenerator.h:74`), so the pass widens the `_Bool` to `int` before storing it; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_size_t` | `non_det_log_t *map2check_save_nondet_log_size_t(int line, int scope, int type_var, size_t *value, const char *function_name)` | — | new | Develop `map2check_nondet_size_t` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SIZE_T (10), low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_loff_t` | `non_det_log_t *map2check_save_nondet_log_loff_t(int line, int scope, int type_var, long long *value, const char *function_name)` | — | new | Develop `map2check_nondet_loff_t` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type LOFF_T (11); `loff_t` is glibc's `long long`; low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_sector_t` | `non_det_log_t *map2check_save_nondet_log_sector_t(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_sector_t` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SECTOR_T (12); `sector_t` is `unsigned long` (`NonDetGenerator.h:112`); low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_pointer` | `non_det_log_t *map2check_save_nondet_log_pointer(int line, int scope, int type_var, void **value, const char *function_name)` | — | new | Develop `map2check_nondet_pointer` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type POINTER (2); the address, logged as its low 32 bits (the pass's `ptrtoint` to `i32`); see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | -| `map2check_save_nondet_log_pchar` | `non_det_log_t *map2check_save_nondet_log_pchar(int line, int scope, int type_var, char **value, const char *function_name)` | — | new | Develop `map2check_nondet_pchar` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type PCHAR (9); the string's address, logged as its low 32 bits; see “Nondet generators and the nondet log”. `type_var` is only kept in the returned record. Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_unsigned` | `non_det_log_t *map2check_save_nondet_log_unsigned(int line, int scope, int type_var, unsigned int *value, const char *function_name)` | — | new | Develop `map2check_nondet_unsigned` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UNSIGNED (5), value printed with `%u`; see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_ulong` | `non_det_log_t *map2check_save_nondet_log_ulong(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_ulong` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type ULONG (6), low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_uchar` | `non_det_log_t *map2check_save_nondet_log_uchar(int line, int scope, int type_var, unsigned char *value, const char *function_name)` | — | new | Develop `map2check_nondet_uchar` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type UCHAR (8), value zero-extended; see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_short` | `non_det_log_t *map2check_save_nondet_log_short(int line, int scope, int type_var, short *value, const char *function_name)` | — | new | Develop `map2check_nondet_short` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SHORT (14), value zero-extended as the pass does (`-2` is logged as `65534`); see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_ushort` | `non_det_log_t *map2check_save_nondet_log_ushort(int line, int scope, int type_var, unsigned short *value, const char *function_name)` | — | new | Develop `map2check_nondet_ushort` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type USHORT (3), value zero-extended; see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_bool` | `non_det_log_t *map2check_save_nondet_log_bool(int line, int scope, int type_var, int *value, const char *function_name)` | — | new | Develop `map2check_nondet_bool` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type BOOL (7); develop's `bool` is `typedef int bool` (`NonDetGenerator.h:74`), so the pass widens the `_Bool` to `int` before storing it; see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_size_t` | `non_det_log_t *map2check_save_nondet_log_size_t(int line, int scope, int type_var, size_t *value, const char *function_name)` | — | new | Develop `map2check_nondet_size_t` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SIZE_T (10), low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_loff_t` | `non_det_log_t *map2check_save_nondet_log_loff_t(int line, int scope, int type_var, long long *value, const char *function_name)` | — | new | Develop `map2check_nondet_loff_t` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type LOFF_T (11); `loff_t` is glibc's `long long`; low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_sector_t` | `non_det_log_t *map2check_save_nondet_log_sector_t(int line, int scope, int type_var, unsigned long *value, const char *function_name)` | — | new | Develop `map2check_nondet_sector_t` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type SECTOR_T (12); `sector_t` is `unsigned long` (`NonDetGenerator.h:112`); low 32 bits with `%d`; see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_pointer` | `non_det_log_t *map2check_save_nondet_log_pointer(int line, int scope, int type_var, void **value, const char *function_name)` | — | new | Develop `map2check_nondet_pointer` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type POINTER (2); the address, logged as its low 32 bits (the pass's `ptrtoint` to `i32`); see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | +| `map2check_save_nondet_log_pchar` | `non_det_log_t *map2check_save_nondet_log_pchar(int line, int scope, int type_var, char **value, const char *function_name)` | — | new | Develop `map2check_nondet_pchar` (`NonDetLog.c:105-150`), called by the pass with the value stored in an `alloca` of its C type (#6, #13). Row type PCHAR (9); the string's address, logged as its low 32 bits; see “Nondet generators and the nondet log”. `type_var` is ignored for the row type (see “Nondet generators and the nondet log”). Returned record is not linked (SD-10). | | `map2check_save_basic_block_log` | `bbtrack_log_t *map2check_save_basic_block_log(int32_t line, const char *function_name)` | returns `bbtrack_log_t *` | same | Returned record is not linked into a list (SD-10). | | `map2check_is_in_trackbb_container` | `int32_t map2check_is_in_trackbb_container(int32_t line)` | `map2check_is_in_trackbb_container` | same | | | `map2check_map_alloca` | `void map2check_map_alloca(const char *var_name, const void *ptr_address, int32_t size, int32_t size_primitive, int32_t line_number, int32_t scope)` | `container_memtracklog.h:41` | same | | @@ -124,7 +124,7 @@ build with `--features libfuzzer`. | `__VERIFIER_nondet_loff_t` | `int64_t __VERIFIER_nondet_loff_t(void)` | `map2check_gen_data_fuzzer_loff_t` | new | Feature `libfuzzer`. Core target of the pass's `map2check_non_det_loff_t` (#6); `loff_t` is glibc's `long long`, eight bytes (SD-18). | | `__VERIFIER_nondet_sector_t` | `unsigned long __VERIFIER_nondet_sector_t(void)` | `map2check_gen_data_fuzzer_sector_t` | new | Feature `libfuzzer`. Core target of `map2check_non_det_sector_t` (#6); develop's `sector_t` is `unsigned long` (`NonDetGenerator.h:112`), eight bytes (SD-18). | | `__VERIFIER_nondet_pchar` | `char *__VERIFIER_nondet_pchar(void)` | `map2check_gen_data_fuzzer_pchar` | new | Feature `libfuzzer`. Core target of `map2check_non_det_pchar` (#6). An `unsigned` length, NULL for 0, capped at 4096, then `length - 1` chars and a NUL, as `NonDetGeneratorAFL.c:101-118`; the string comes from the C `malloc` and the program may `free` it (SD-18). | -| `__VERIFIER_assume` | `void __VERIFIER_assume(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assume(0)` abandons the current fuzzer input and `LLVMFuzzerTestOneInput` returns 0, as develop's `nondet_cancel`; with no iteration running it calls `exit(0)` (SD-07). Defined `extern "C-unwind"`; see “`__VERIFIER_assume` under a fuzzer”. | +| `__VERIFIER_assume` | `void __VERIFIER_assume(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assume(0)` abandons the current fuzzer input and `LLVMFuzzerTestOneInput` returns 0, as develop's `nondet_cancel`; with no iteration running it calls `exit(0)` (SD-07). Defined `extern "C-unwind"`. The analysed program must be compiled with `-fexceptions` (D1 condition 11); see “`__VERIFIER_assume` under a fuzzer”. | | `map2check_fuzzer_assume` | `void map2check_fuzzer_assume(int32_t condition)` | `map2check_fuzzer_assume(int)` (`nondet_gen_libfuzzer.h:41`) | changed | Feature `libfuzzer`. Same behaviour as `__VERIFIER_assume` (SD-07). | | `__VERIFIER_error` | `void __VERIFIER_error(void)` | — | new | Feature `libfuzzer`. Records REACHABILITY, prints the JSON report when the mode is unset, then aborts (SD-08). | | `__VERIFIER_assert` | `void __VERIFIER_assert(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assert(0)` calls `__VERIFIER_error()` (SD-08). | @@ -291,7 +291,18 @@ number of the export (`Map2CheckTypes.h:41-58`), and the value as types zero-extended, wide types and addresses truncated to 32 bits), printed with `%u` for `unsigned`, `%lf` for `double` and `%d` otherwise (`NonDetLog.c:45-54`). The golden `tests/legacy_golden/NONDET-TYPES/` holds -one row of each type. `type_var` is only kept in the returned record. +one row of each type. + +**`type_var` is ignored for the row type.** It is only copied into the +returned record. The row type comes from the export, as in develop, whose +wrappers are generated per type (`MAP2CHECK_NONDET_GENERATOR(type, +nondet_type)`, `NonDetLog.c:105-128`) and take no type argument. Trusting +`type_var` would let one call carry two types, the export's C type for the +pointer and `type_var` for the row; deriving it from the export keeps them +the same. The cost: a pass that calls the wrong saver logs the wrong type +silently, so Task 3.2 must map each `map2check_nondet_` to its +`map2check_save_nondet_log_` exactly as `docs/cutover/abi-mapping.tsv` +says. ## `__VERIFIER_assume` under a fuzzer @@ -307,10 +318,43 @@ assume calls `std::panic::resume_unwind` with a private payload (no panic hook runs, nothing is printed), and `LLVMFuzzerTestOneInput` catches it with `catch_unwind` and returns 0. The exports and the `__map2check_main__` import are `extern "C-unwind"`, the ABI that lets a Rust unwind cross C -frames (RFC 2945). The analysed program's frames in between must carry -unwind tables, which clang emits by default on x86-64 Linux; they hold no -cleanups, so the unwind skips exactly what develop's `longjmp` skips. The -unwind needs `panic = "unwind"` (the `release` and `runtime` profiles). +frames (RFC 2945). The unwind needs `panic = "unwind"` (the `release` and +`runtime` profiles). + +**The analysed program must be compiled with `-fexceptions` (D1 condition +11, Task 3.2).** clang marks every C function `nounwind` unless it gets +`-fexceptions`, and LLVM IR makes unwinding out of a `nounwind` function +undefined. Without the flag the unwind works only because x86-64 Linux +emits `.eh_frame` by default; with `-fno-asynchronous-unwind-tables +-fno-unwind-tables` it aborts ("failed to initiate panic, error 5"), which +AFL++ would count as a crash. Measured on `tests/ffi_harness.c` at `-O2`, +with clang-16 and afl-clang-fast 4.40c +(`scripts/cutover/d1-remeasure/results/flags16.txt`): + +| Program flags | `__map2check_main__` attributes | Harness | +|---|---|---| +| default | `nounwind uwtable` | passes (undefined in IR) | +| `-funwind-tables` | `nounwind uwtable` | passes (undefined in IR) | +| `-fno-asynchronous-unwind-tables -fno-unwind-tables` | `nounwind` | aborts | +| `-fexceptions` | `uwtable` | passes, defined | +| `-fexceptions -fno-asynchronous-unwind-tables -fno-unwind-tables` | (none) | passes, defined | + +`-fexceptions` is the required flag. It is the only one that removes +`nounwind`, and it keeps the unwind information even if a later flag drops +the tables. It must be given where the program becomes bitcode (the +`clang -emit-llvm` step), because the attributes travel in the bitcode +that `afl-clang-fast` receives. Unwind tables alone are not enough: they +make the unwind work, not defined. CI builds the harness at `-O2` with and +without the flag. + +Plain C frames hold no cleanups, so the unwind skips what develop's +`longjmp` skips. A frame with `__attribute__((cleanup))` compiled with +`-fexceptions` would have its cleanup run, which `longjmp` would skip. + +`ITERATION_ACTIVE`, the flag that tells an assume whether an iteration is +running, is process-wide: an assume on another thread while an iteration +runs aborts the process (develop's cross-thread `longjmp` is undefined +too). With no iteration running (no `LLVMFuzzerTestOneInput` on the stack), or in a `panic = "abort"` build, the assume keeps ending the process with diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 0b1c990..657a5f2 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -245,8 +245,12 @@ map2check_check_free("p", (void *)0x5000, 0, 4, "main"); /* Memsafet frames back to `LLVMFuzzerTestOneInput`, which catches it and returns 0. The fuzzer goes on with the next input in the same process, and the next iteration starts from an empty state (SD-11). Both exports, and the -`__map2check_main__` import, are `extern "C-unwind"`; the program's frames -need unwind tables, the clang default on x86-64 Linux. With no iteration +`__map2check_main__` import, are `extern "C-unwind"`. The program must be +compiled with `-fexceptions` (D1 condition 11): without it clang marks the +C frames `nounwind`, and the unwind works only thanks to the default +`.eh_frame`, undefined in LLVM IR. A program frame with +`__attribute__((cleanup))` compiled with `-fexceptions` has its cleanup run +by the unwind, where develop's `longjmp` skips it. With no iteration running (a program linked without a fuzzer driver), or in a `panic = "abort"` build, they keep calling `exit(0)`. diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index 651f49f..3f173f7 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -490,8 +490,11 @@ macro_rules! nondet_ffi { // One export per `map2check_nondet_` of develop (`NonDetLog.c:105-150`), // each taking a pointer to the value in the C type `NonDetGenerator.h` gives -// that family. The `type_var` argument is only kept in the returned record: -// the `enum NONDET_TYPE` of the row comes from the export, as in develop. +// that family. The `type_var` argument is ignored for the row type and only +// kept in the returned record: the `enum NONDET_TYPE` of the row comes from +// the export, as develop's per-type wrappers do (`NonDetLog.c:105-128`), so +// the value's C type and the row type cannot disagree. A pass that calls the +// wrong saver therefore logs the wrong type (docs/api-deltas.md). nondet_ffi!(map2check_save_nondet_log_int, i32, i, 0, NonDetValue::Int); nondet_ffi!(map2check_save_nondet_log_uint, u32, u, 0, NonDetValue::UInt); nondet_ffi!( diff --git a/rust/src/nondet/libfuzzer.rs b/rust/src/nondet/libfuzzer.rs index 56c8048..8150cab 100644 --- a/rust/src/nondet/libfuzzer.rs +++ b/rust/src/nondet/libfuzzer.rs @@ -26,6 +26,12 @@ static FUZZER_CURSOR: AtomicUsize = AtomicUsize::new(0); static FUZZER_SIZE: AtomicUsize = AtomicUsize::new(0); static FUZZER_DATA: AtomicPtr = AtomicPtr::new(std::ptr::null_mut()); /// True while `LLVMFuzzerTestOneInput` runs the program for one input. +/// +/// Process-wide, not per thread: a program thread that calls +/// `__VERIFIER_assume(0)` while an iteration runs on another thread unwinds +/// a stack with no `catch_unwind` on it, and the process aborts. Develop's +/// cross-thread `longjmp` is undefined as well; the fuzzer drives the +/// program on one thread. static ITERATION_ACTIVE: AtomicBool = AtomicBool::new(false); struct FuzzerStateReset; @@ -258,10 +264,17 @@ pub extern "C" fn __VERIFIER_nondet_pchar() -> *mut std::os::raw::c_char { // Every Rust function on that path is declared `C-unwind` // (`__VERIFIER_assume`, `map2check_fuzzer_assume`, the `__map2check_main__` // import), which is the defined way for a Rust unwind to cross C frames -// (RFC 2945). The C frames in between are the analysed program's: they -// must carry unwind tables, which clang emits by default on x86-64 Linux -// (`-funwind-tables`); they hold no cleanups, so nothing is skipped that -// develop's `longjmp` would not skip as well. The unwind needs +// (RFC 2945). The C frames in between are the analysed program's. clang +// marks C functions `nounwind` unless the program is compiled with +// `-fexceptions`, and unwinding out of a `nounwind` function is undefined +// in LLVM IR; without `-fexceptions` the unwind only works because x86-64 +// Linux emits `.eh_frame` by default, and with no unwind tables at all it +// aborts ("failed to initiate panic"). D1 condition 11 therefore requires +// the pipeline to compile the program with `-fexceptions` (Task 3.2). Plain +// C frames hold no cleanups, so the unwind skips what develop's `longjmp` +// skips; a frame with `__attribute__((cleanup))` compiled with +// `-fexceptions` would have its cleanup run, which `longjmp` would skip. +// The unwind needs // `panic = "unwind"` (the `runtime` and `release` profiles); built with // `panic = "abort"` the library keeps the exit(0) fallback below. // From 670758e7dbaea58b029167c0425ed72ec657f00b Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 09:20:16 -0400 Subject: [PATCH 37/63] docs: record the maintainer's D1 decision to mitigate in Rust Condition 3 stays NOT MET with its numbers and is replaced by gate 4.2's criteria (median wall-time ratio <= 1.10, no new timeouts). Condition 4 is mitigated by a frontend seed of at least 8 bytes (Task 3.3, rechecked at gate 4.2). New condition 11: the program is compiled with -fexceptions (Task 3.2). The plan gets the matching steps in Tasks 3.2 and 3.3, and the spike write-up records corpus_count and the B2 queue entries behind the condition 4 analysis. --- docs/cutover/decisoes.md | 11 ++++++++--- docs/cutover/spike-llvm16.md | 6 +++--- .../plans/2026-10-15-cutover-runtime-c-map2check.md | 2 ++ 3 files changed, 13 insertions(+), 6 deletions(-) diff --git a/docs/cutover/decisoes.md b/docs/cutover/decisoes.md index dbd697f..4239bb0 100644 --- a/docs/cutover/decisoes.md +++ b/docs/cutover/decisoes.md @@ -15,7 +15,7 @@ Evidência: `docs/cutover/spike-llvm16.md` (Task 1.1). Código na branch local ` | Gerador | Rota proposta | Evidência principal | |---|---|---| | `none` | **B**: `libmap2check.a` nativo (rustc do repo, 1.82), ligado no passo final, sem `llvm-link` | A rota B liga e roda com clang-16 (`rc=0`). O frontend de `develop` não executa `none` como gerador. `Caller::executeAnalysis` registra "This mode is not supported", e `map2check.cpp:977-1008` troca `None` por AFL++ → KLEE (→ AFL++ de novo com `--seed-exchange`). Na prática, `none` é a combinação de `afl` e `symex`, e herda as condições das duas, inclusive a 9. | -| `afl` | **B + B1**: `libmap2check.a` nativo + driver libFuzzer do AFL++ (`afl-clang-fast -fsanitize=fuzzer`), chamando o `LLVMFuzzerTestOneInput` da biblioteca | Os dois fuzzers acharam `x == 42` em 30 s, com 1 `saved_crashes` cada. Throughput: B1 97 474 exec/s, contra 42 919 do `NonDetGeneratorAFL.c` (B2), ou 227% (229% com CmpLog). **O critério da D1 (≥ 90% do C) está atendido provisoriamente.** A comparação não é igual por igual (condição 3). Com CmpLog, que é como o frontend sempre roda, o B1 levou **3 037 ms / 280 900 execs** até o crash, contra **252 ms / 2 748 execs** do B2 (condição 4). O B2, port do `__AFL_LOOP` para Rust, continua como alternativa nomeada se as condições 3 ou 4 falharem. | +| `afl` | **B + B1**: `libmap2check.a` nativo + driver libFuzzer do AFL++ (`afl-clang-fast -fsanitize=fuzzer`), chamando o `LLVMFuzzerTestOneInput` da biblioteca | Os dois fuzzers acharam `x == 42` em 30 s, com 1 `saved_crashes` cada. Throughput: B1 97 474 exec/s, contra 42 919 do `NonDetGeneratorAFL.c` (B2), ou 227% (229% com CmpLog). **O critério da D1 (≥ 90% do C) está atendido provisoriamente.** A comparação não é igual por igual (condição 3). Com CmpLog, que é como o frontend sempre roda, o B1 levou **3 037 ms / 280 900 execs** até o crash, contra **252 ms / 2 748 execs** do B2 (condição 4). O B2, port do `__AFL_LOOP` para Rust, continua como alternativa nomeada se as condições 3 ou 4 falharem. Remedição da Task 2.5b: 3 e 4 não atendidas; o mantenedor decidiu mitigar em Rust (2026-10-16): a 3 passa a ser julgada pelo gate 4.2 e a 4 é mitigada pela semente de ≥ 8 bytes (condições 3, 4 e 11). | | `symex` | **A′**: runtime compilado para bitcode por um rustc com LLVM 16 (1.72.1, LLVM 16.0.5), com manifesto e lockfile próprios, LTO fat e `opt-16 --strip-debug` só no runtime, depois `llvm-link-16` com o bitcode do programa | A rota A falha: `llvm-dis-16: error: Unknown attribute kind (91) (Producer: 'LLVM19.1.1-rust-1.82.0-stable' Reader: 'LLVM 16.0.6')`. A rota B sob KLEE dá 1 caminho e não explora `x == 42`. A A′ dá 2 caminhos (ktests 42 e 0) com a crate `no_std` do brief. A **biblioteca real** (`std`, `serde_json`, `Mutex`) roda no KLEE 3.1 com um subconjunto das flags do `caller.cpp`: 22 caminhos completos, `x == 42` encontrado. | A rota C (Map2Check em LLVM 19 + KLEE 3.3) fica como **saída futura do pin do rustc 1.72.1**, não como caminho do cutover (ver "Custo do pin e saída"). @@ -125,17 +125,22 @@ A aprovação aceita a proposta junto com estas condições. A remedição exige a saída em arquivo do Rust (Task 2.4), um `__VERIFIER_assume` que não encerre o processo persistente e programas com muitos `assume`. O Rust faz `exit(0)` (`libfuzzer.rs:197-206`); o C faz `longjmp` e segue para a próxima entrada. As campanhas devem rodar em sequência, com as mesmas flags e pelo menos 5 repetições. Abaixo de 90% do C, o `afl` passa para B2 (port do `__AFL_LOOP`). - **Status (Task 2.5b, 2026-10-15): NÃO ATENDIDA.** Remedido em sequência, 5 repetições, `--cpus 4`, B2 compilado das fontes de `develop` e gravando arquivos, `assume` que abandona a entrada sem encerrar o processo (`spike-llvm16.md`, “Remedição (Task 2.5b)”). Medianas de exec/s, B1/B2: `prog42` 116 390 / 98 060 (**119%**) sem CmpLog e 115 510 / 97 434 (119%) com CmpLog; programa com `assume` 64 018 / 115 493 (**55%**) sem CmpLog e 75 394 / 107 069 (**70%**) com CmpLog. A perda vem do desenrolar de pilha que rejeita cada entrada (o B2 usa `longjmp`). A rota não foi alterada; a decisão fica com o controlador. + **Status: NÃO ATENDIDA — substituída pelo gate 4.2 por decisão do mantenedor em 2026-10-16.** Medição da Task 2.5b (2026-10-15): Remedido em sequência, 5 repetições, `--cpus 4`, B2 compilado das fontes de `develop` e gravando arquivos, `assume` que abandona a entrada sem encerrar o processo (`spike-llvm16.md`, “Remedição (Task 2.5b)”). Medianas de exec/s, B1/B2: `prog42` 116 390 / 98 060 (**119%**) sem CmpLog e 115 510 / 97 434 (119%) com CmpLog; programa com `assume` 64 018 / 115 493 (**55%**) sem CmpLog e 75 394 / 107 069 (**70%**) com CmpLog. A perda vem do desenrolar de pilha que rejeita cada entrada (o B2 usa `longjmp`). A rota não foi alterada. + + **Decisão do mantenedor (2026-10-16): “Mitigar em Rust”, sem trecho C.** O throughput sintético deixa de bloquear sozinho. Ele é substituído pelos critérios do gate 4.2 no corpus real: razão mediana de tempo de parede (B1 ÷ C) **≤ 1,10** e **nenhum timeout novo**. Os números acima ficam registrados como medidos. 4. **O tempo até o primeiro crash com CmpLog é critério de aceitação da D1.** Ele será remedido no corpus no gate 4.2. O frontend sempre roda o afl-fuzz com `-c `, então para guardas de valor mágico o que conta é o tempo até o crash, não exec/s. No spike, com CmpLog, o B1 levou cerca de 100× mais execuções que o B2 (280 900 contra 2 748), e o B2 achou o crash pela etapa de input-to-state (`op:its`). **Hipótese principal,** tirada do código e não medida (o spike não foi refeito): a diferença está em entradas mais curtas que o valor lido. O `read_bytes` do Rust (`libfuzzer.rs:54-71`) devolve `None` se faltarem bytes, e `read_u32` vira **0**. O C (`get_next_input_from_afl`) **volta ao início do buffer**. Com a semente `A` (1 byte), o C lê `x = 0x41414141`, que aparece na entrada. O CmpLog vê `0x41414141 == 42`, acha os bytes e os troca por `2a 00 00 00`. No Rust, `x = 0` não vem de nenhum byte da entrada, e o input-to-state não tem o que substituir. Só a mutação havoc, depois de crescer a entrada para 4 bytes, chega a `x = 42`. Hipótese secundária, do revisor: a leitura passa por código Rust não instrumentado. A correção provável da principal é a Task 2.5 alinhar a semântica de entrada curta à do C. Isso também é exigido pela condição 9. - **Status (Task 2.5b, 2026-10-15): NÃO ATENDIDA.** Com o leitor já igual ao do C, mediana de 5 campanhas `prog42` com CmpLog: B1 **822 ms** (86 511 execs, sempre `op:havoc`), B2 **190 ms** (544 execs, sempre `op:its`), ou 4,33× (o limite é 2×). Antes da correção do leitor (`0d5b1f6`), o B1 tinha mediana de 749 ms: a hipótese principal não explica a diferença. Vale a secundária: o runtime C instrumentado gera arestas (o ramo de volta ao início do buffer) que põem entradas de 4 bytes no corpus, onde o input-to-state troca o operando; o leitor Rust não é instrumentado e o corpus do B1 fica só com a semente `A`, de 1 byte. Com a semente `AAAA`, B1 e B2 acham o crash por `op:its` em cerca de 175 ms. No programa com `assume`, com CmpLog, a condição é atendida (B1 145 ms, B2 298 ms). A rota não foi alterada; a decisão fica com o controlador. + **Status: NÃO ATENDIDA na medição da Task 2.5b (2026-10-15); mitigada por decisão do mantenedor em 2026-10-16.** Com o leitor já igual ao do C, mediana de 5 campanhas `prog42` com CmpLog: B1 **822 ms** (86 511 execs, sempre `op:havoc`), B2 **190 ms** (544 execs, sempre `op:its`), ou 4,33× (o limite é 2×). Antes da correção do leitor (`0d5b1f6`), o B1 tinha mediana de 749 ms: a hipótese principal não explica a diferença. Vale a secundária: o runtime C instrumentado gera arestas (o ramo de volta ao início do buffer) que põem entradas de 4 bytes no corpus, onde o input-to-state troca o operando; o leitor Rust não é instrumentado e o corpus do B1 fica só com a semente `A`, de 1 byte. Com a semente `AAAA`, B1 e B2 acham o crash por `op:its` em cerca de 175 ms. No programa com `assume`, com CmpLog, a condição é atendida (B1 145 ms, B2 298 ms). A rota não foi alterada. + + **Mitigação (decisão do mantenedor, 2026-10-16, “Mitigar em Rust”):** o frontend dá ao AFL++ uma semente inicial de **pelo menos 8 bytes**, o tamanho do maior valor lido (`long`, `double`, ponteiros). Com isso o input-to-state tem o operando inteiro na entrada desde a primeira execução, sem depender de o corpus crescer por arestas do runtime. Responsável: Task 3.3, onde o frontend cria o diretório `-i` do afl-fuzz (`caller.cpp` ~893-905 em `develop`), com teste em `CallerLink` de que o arquivo de semente tem ≥ 8 bytes. Remedido no gate 4.2, no corpus real. 5. **MSRV 1.72 no que o build `symex` compila,** com os custos (a)–(c) acima e o `clear_poison` sob `#[cfg(not(feature = "klee"))]`. Responsável: Task 2.7, com o job de CI em 1.72.1. 6. **LTO fat e `--strip-debug` do runtime `symex` são obrigatórios.** O `-g` do programa analisado é preservado. Responsável: Task 2.7. 7. **Concretizar antes de serializar no `symex`.** O `serde_json` formatando valores simbólicos levou o `prog.c` de 2 para 22 caminhos. O gerador KLEE do Rust precisa concretizar (`klee_get_value_*`) o que vai para o log e para o relatório, como o C faz ao escrever por chamada externa. Responsável: Task 2.7. 8. **A rota B sob KLEE não é usada.** Além de não explorar símbolos, o KLEE 3.1 não tem `--load` (`Unknown command line argument '--load=...'`), e a `.so` só carrega com `LD_PRELOAD` e `-Wl,-z,lazy`. 9. **O layout do vetor tem de ser o mesmo entre KLEE e AFL++.** No modo padrão (`none` → AFL++ → KLEE → AFL++ com `--seed-exchange`), os vetores do KLEE viram sementes do AFL++ (`Caller::exportKleeVectorsAsSeeds`). O gerador AFL do Rust tem de consumir a semente exatamente como o KLEE a produz: `sizeof(tipo)` bytes por valor, em ordem, little-endian. É o que `read_u32` faz hoje para `int`, mas falta verificar os demais tipos. A semântica de entrada curta precisa estar alinhada com a do C (condição 4). Responsável: Tasks 2.5 e 2.7, com teste cruzado ktest → semente → replay. 10. **O rustc 1.72.1 (2023) fica preso ao LLVM 16.** Quando o Map2Check migrar de LLVM, o toolchain do `symex` precisa acompanhar ou ser removido pela rota C. +11. **O programa analisado é compilado com `-fexceptions`.** Responsável: Task 3.2. Um `__VERIFIER_assume` falso dentro de uma iteração do fuzzer desenrola a pilha (`resume_unwind`) através dos frames C do programa até o `LLVMFuzzerTestOneInput`. Sem `-fexceptions`, o clang marca toda função C como `nounwind`, e desenrolar a partir de uma função `nounwind` é comportamento indefinido no IR do LLVM. Hoje funciona só porque o x86-64 Linux emite `.eh_frame` por padrão. Com `-fno-asynchronous-unwind-tables -fno-unwind-tables` o processo aborta (“failed to initiate panic, error 5”), e o AFL++ contaria isso como crash: um falso positivo. O passo que gera o bitcode do programa (`clang -emit-llvm`) tem de passar `-fexceptions`, porque os atributos viajam no bitcode que o `afl-clang-fast` recebe. Nenhuma etapa pode acrescentar `-fno-unwind-tables` ou `-fno-asynchronous-unwind-tables`. Unwind tables sozinhas (`-funwind-tables`) não bastam: fazem o desenrolar funcionar, mas o IR continua `nounwind`. Evidência: `tests/ffi_harness.c` em `-O2` com clang-16 e afl-clang-fast (`scripts/cutover/d1-remeasure/results/flags16.txt`, tabela em `docs/api-deltas.md`, “`__VERIFIER_assume` under a fuzzer”). O CI roda o harness em `-O2` com e sem `-fexceptions`. diff --git a/docs/cutover/spike-llvm16.md b/docs/cutover/spike-llvm16.md index ffa3d6e..1fd59bc 100644 --- a/docs/cutover/spike-llvm16.md +++ b/docs/cutover/spike-llvm16.md @@ -429,7 +429,7 @@ AFL_CRASHING_SEEDS_AS_NEW_CRASH=1 afl-fuzz -i seed -o runs/---< [-c ./-.cmplog.afl] -- ./-.afl ``` - Ordem: para cada repetição, `prog42` e depois `assume`; sem e com CmpLog; B1 e depois B2. O 1º crash vem do nome do primeiro arquivo em `crashes/` (`time`, `execs`, `op`). + Ordem: para cada repetição, `prog42` e depois `assume`; sem e com CmpLog; B1 e depois B2. O 1º crash vem do nome do primeiro arquivo em `crashes/` (`time`, `execs`, `op`). Scripts, programas e resultados brutos estão em `scripts/cutover/d1-remeasure/` (README com os comandos de docker). Antes das campanhas, um teste diferencial ligou um programa que lê um valor de cada tipo (16 tipos, `pchar` incluído) ao B1 e ao B2 e comparou as saídas para entradas de 1, 2, 4, 7, 7 (`pchar` de tamanho 5), 37, 64 e 200 bytes: **idênticas em todas** (B2 lendo da stdin, B1 pelo `LLVMFuzzerTestOneInput`). @@ -493,7 +493,7 @@ Todas as 40 campanhas acharam o crash. Estabilidade: B1 100% em todas; B2 75,68% **Throughput (condição 3).** No `prog42` o B1 fica em 119% do B2, agora com o B2 gravando arquivos e as campanhas em sequência. No programa com `assume`, quase toda entrada é rejeitada, e o B1 cai para **55%** (sem CmpLog) e **70%** (com CmpLog) do B2. O B2 rejeita com `longjmp`, que custa poucos nanossegundos; o B1 rejeita com um desenrolar de pilha (`resume_unwind` + `catch_unwind`), que passa pelo unwinder (`_Unwind_RaiseException`, busca de FDE) a cada rejeição. O B1 também varia mais entre repetições (57 884 a 100 495 exec/s sem CmpLog), o que é consistente com o custo crescer com a fração de entradas rejeitadas. -**Tempo até o 1º crash com CmpLog (condição 4).** No `prog42` o B2 achou o crash pela etapa input-to-state (`op:its`) nas 5 repetições, em 185 a 207 ms; o B1, por havoc nas 5, com mediana de 822 ms. Corrigir o leitor (a hipótese principal) **não fechou a diferença**. Linha de base com a biblioteca de antes desta task (`0d5b1f6`, leitor que devolvia 0), mesmo protocolo, só `prog42` com CmpLog: 102, 749, 2 845, 115 e 1 083 ms, mediana **749 ms**, todas por havoc. A diferença está no corpus. No B2, o runtime C é instrumentado, e o ramo de volta ao início do buffer de `get_next_input_from_afl` é uma aresta nova: entradas de 2 e 4 bytes (`41 41`, `c0 c0 c0 c0`) entram no corpus em 50 execuções (`corpus_count` 4), e o input-to-state troca os 4 bytes do `c0c0c0c0` por `2a 00 00 00`. No B1 o leitor é Rust não instrumentado, o corpus fica só com a semente `A` (`corpus_count` 1), e o input-to-state não consegue pôr um operando de 4 bytes numa entrada de 1 byte. É a hipótese secundária da condição 4. +**Tempo até o 1º crash com CmpLog (condição 4).** No `prog42` o B2 achou o crash pela etapa input-to-state (`op:its`) nas 5 repetições, em 185 a 207 ms; o B1, por havoc nas 5, com mediana de 822 ms. Corrigir o leitor (a hipótese principal) **não fechou a diferença**. Linha de base com a biblioteca de antes desta task (`0d5b1f6`, leitor que devolvia 0), mesmo protocolo, só `prog42` com CmpLog: 102, 749, 2 845, 115 e 1 083 ms, mediana **749 ms**, todas por havoc. A diferença está no corpus. No B2, o runtime C é instrumentado, e o ramo de volta ao início do buffer de `get_next_input_from_afl` é uma aresta nova: entradas de 2 e 4 bytes (`41 41`, `c0 c0 c0 c0`) entram no corpus em 50 execuções (`corpus_count` 4), e o input-to-state troca os 4 bytes do `c0c0c0c0` por `2a 00 00 00`. No B1 o leitor é Rust não instrumentado, o corpus fica só com a semente `A` (`corpus_count` 1), e o input-to-state não consegue pôr um operando de 4 bytes numa entrada de 1 byte. É a hipótese secundária da condição 4. Registro por campanha em `scripts/cutover/d1-remeasure/results/corpus-v1.tsv`: no `prog42`, `corpus_count` 1 nas 10 campanhas do B1 e 4 nas 10 do B2. O 1º crash do B2 com CmpLog veio de `src:000002` em 4 repetições e de `src:000001` na 5ª, sempre uma entrada de 4 bytes (`c0c0c0c0`, `ffffffff`, `7a7a7a7a`, `41414141`, `7c7c7c7c`). O do B1 veio sempre de `src:000000`, a semente. Fica uma ressalva: a estabilidade de 75-78% do B2 mostra arestas variáveis no runtime C instrumentado, e são arestas assim, não necessariamente só o ramo de volta ao início, que fazem o corpus do B2 crescer. A causa da queda da condição 3 (o custo do desenrolar por rejeição) é inferida, não isolada: não houve perfil nem microbenchmark. Verificação (informativa, não altera o protocolo): a mesma campanha `prog42` com CmpLog, mas com a semente `AAAA` e `AFL_BENCH_UNTIL_CRASH=1`, 5 repetições sequenciais. O B1 achou o crash em 171 a 184 ms (13 execs, `op:its`) e o B2 em 170 a 184 ms (25 execs, `op:its`): as duas rotas ficam iguais quando a semente já tem o tamanho do valor lido. @@ -504,7 +504,7 @@ No programa com `assume`, com CmpLog, o B1 achou o crash antes do B2 (mediana 14 - **Condição 3 (throughput ≥ 90% do B2): NÃO ATENDIDA.** Atendida no `prog42` (119%), não atendida no programa com `assume` (55% sem CmpLog, 70% com CmpLog). - **Condição 4 (1º crash com CmpLog não pior que 2× a mediana do B2): NÃO ATENDIDA** no `prog42` (822 ms contra 190 ms, 4,33×); atendida no programa com `assume` (0,49×). -A rota da D1 não foi alterada. Caminhos possíveis, para decisão do controlador: (a) para a condição 4, uma semente no frontend (Task 3.x) com pelo menos o tamanho do maior valor lido (8 bytes cobrem `long`, `double` e ponteiros); com 4 bytes e um `int`, o teste acima igualou B1 e B2; (b) para a condição 3, rejeitar a entrada sem desenrolar a pilha, por exemplo com um `setjmp` num trecho C ligado à biblioteca (exige compilar C no `build.rs`, e o `longjmp` só é definido se nenhum frame Rust pulado tiver destrutor); (c) a alternativa nomeada na D1, o B2 (port do `__AFL_LOOP`). +**Decisão do mantenedor (2026-10-16): “Mitigar em Rust”**, sem trecho C. A condição 3 passa a ser julgada pelos critérios do gate 4.2 (razão mediana de tempo de parede ≤ 1,10, nenhum timeout novo). A 4 é mitigada pela semente de ≥ 8 bytes no frontend (Task 3.3). Ver `decisoes.md`, condições 3, 4 e 11. Os caminhos que estavam em aberto eram: (a) para a condição 4, uma semente no frontend (Task 3.x) com pelo menos o tamanho do maior valor lido (8 bytes cobrem `long`, `double` e ponteiros); com 4 bytes e um `int`, o teste acima igualou B1 e B2; (b) para a condição 3, rejeitar a entrada sem desenrolar a pilha, por exemplo com um `setjmp` num trecho C ligado à biblioteca (exige compilar C no `build.rs`, e o `longjmp` só é definido se nenhum frame Rust pulado tiver destrutor); (c) a alternativa nomeada na D1, o B2 (port do `__AFL_LOOP`). ## Como reproduzir diff --git a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md index df984ca..d64ca6d 100644 --- a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md +++ b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md @@ -1124,6 +1124,7 @@ Escrever um teste por hook do enum (28), mais `nonDetGenerator`/`nonDetLogger` p - [ ] **Step 3: Rodar e ver falhar.** Run: `cmake --build build && cd build && ctest -R 'RuntimeABI|PassIR' --output-on-failure` → falha de compilação (`RuntimeABI.hpp` inexistente). - [ ] **Step 4: Implementar a tabela e migrar um passe por commit**, na ordem `TrackBasicBlockPass`, `AssertPass`, `TargetPass`, `LoopPredAssumePass`, `OverflowPass`, `NonDetPass`, `MemoryTrackPass`, `Map2CheckLibrary`. Cada commit deixa os testes `PassIR` do passe migrado verdes. - [ ] **Step 5: Teste de link ponta a ponta.** Para cada input de `tests/unit/pass/inputs/`, instrumentar com todos os passes do modo, ligar a `libmap2check.a` com `clang-16` e garantir zero `undefined reference`. +- [ ] **Step 5b: O programa instrumentado mantém o desenrolar definido (D1, condição 11).** O passo que gera o bitcode do programa analisado (`clang-16 -emit-llvm`, antes dos passes) recebe `-fexceptions`, e nenhuma etapa do pipeline acrescenta `-fno-unwind-tables` ou `-fno-asynchronous-unwind-tables`. Sem `-fexceptions`, o clang marca as funções C `nounwind`, e o `__VERIFIER_assume` que rejeita a entrada desenrola a pilha através delas (comportamento indefinido no IR; aborta se faltarem as tabelas). Teste: um input de `tests/unit/pass/inputs/` com `__VERIFIER_assume`, compilado pelo comando do pipeline, não tem `nounwind` no `define` de `__map2check_main__` (conferir com `llvm-dis-16`); e o binário `-O2` desse input, ligado à `libmap2check.a`, roda 100 entradas rejeitadas seguidas de uma aceita sem abortar. A matriz de flags medida está em `docs/api-deltas.md` (map2check-library), “`__VERIFIER_assume` under a fuzzer”. - [ ] **Step 6: Commit (na branch)** `feat(pass): emit the core runtime API of map2check-library (#23)` **Critério de conclusão:** `RuntimeABI*` e `PassIR*` passam; `grep -rnE '"map2check_(malloc|calloc|free|load|alloca|assert|track_bb|function|posix|binop_(add|sub|mul|sdiv)(_uint)?|non_det_[a-z_]+|nondet_[a-z_]+)"' modules/backend/pass` não retorna nada; o link ponta a ponta não tem símbolos indefinidos. @@ -1164,6 +1165,7 @@ Os enumeradores seguem `modules/frontend/caller.hpp:21-45` de `develop`; incluir - [ ] **Step 2–4:** Ver falhar, extrair as funções e implementar, `ctest --output-on-failure` verde. - [ ] **Step 4b:** Os binários de witness (`-witness-fuzzed.out` e o `-witness-result.bc` que vai ao KLEE) ligam também um objeto gerado pelo frontend com `__attribute__((constructor)) static void m2c_witness(void) { map2check_enable_witness(); }`, compilado com `clang-16 -c` (nativo) ou `-emit-llvm` (KLEE). Os binários de verificação não o ligam. Isso reproduz a escolha `WitnessGeneration.bc` × `WitnessGenerationNone.bc` de `caller.cpp:635-640` (ruling da Task 2.4). Teste em `CallerLink`: o comando de witness contém o objeto e o de verificação não. +- [ ] **Step 4c: Semente inicial de ≥ 8 bytes para o AFL++ (D1, condição 4, mitigação decidida pelo mantenedor em 2026-10-16).** Onde o frontend cria o diretório `-i` do afl-fuzz (`caller.cpp` ~893-905 em `develop`: `afl-in` ou o diretório de troca de sementes, com o placeholder `seed << "A"` quando vazio), a semente placeholder passa a ter pelo menos 8 bytes, o tamanho do maior valor lido pelo gerador (`long`, `double`, ponteiros). Assim o input-to-state do CmpLog acha o operando inteiro na entrada sem depender de o corpus crescer por arestas do runtime C, que o `libmap2check.a` não instrumentado não gera (`docs/cutover/spike-llvm16.md`, “Remedição (Task 2.5b)”: com a semente `AAAA` o B1 acha `x == 42` por `op:its` em ~175 ms, como o B2). Extrair a criação da semente para uma função pura testável. Teste em `CallerLink`: o arquivo de semente que o frontend escreve num diretório vazio tem ≥ 8 bytes, e um diretório que já tem sementes (troca com o KLEE) não é alterado. A condição 4 é remedida no gate 4.2, no corpus real. - [ ] **Step 5:** Se D3 = remover, adicionar o aviso de `--btree` com o seu teste. - [ ] **Step 6: Commit (na branch)** `feat: link map2check-library instead of the C bitcode runtime` From cb95649982abdb050db06d6190597046fdf5e043 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:11:37 -0400 Subject: [PATCH 38/63] fix: compile the shared sources with rustc 1.72 for the KLEE runtime The symex runtime (D1 route A') builds rust/src with rustc 1.72.1, whose LLVM 16 writes bitcode KLEE 3.1 reads (condition 5): - state.rs: OnceLock::get_or_init instead of LazyLock (unstable in 1.72); Mutex::clear_poison stays in the normal build and is compiled out under feature "klee", whose panic = "abort" profile cannot poison the lock. - memtrack: map_or(true, ..) instead of Option::is_none_or (1.82). - The three HashSets use fixed SipHash keys (memtrack::FixedHashSet): RandomState seeds itself through a weak getrandom symbol, which under KLEE is an invalid function pointer. The sets only answer membership. - Cargo.toml: the empty `klee` feature leaves the main manifest; it belongs to the KLEE build's own manifest. Here `--all-features` would enable it next to `libfuzzer` (and drop clear_poison from the tests). The value is declared for check-cfg instead. --- Cargo.toml | 8 +++++++- rust/src/memtrack/mod.rs | 21 ++++++++++++++++----- rust/src/output/legacy_files.rs | 2 +- rust/src/state.rs | 13 +++++++++---- 4 files changed, 33 insertions(+), 11 deletions(-) diff --git a/Cargo.toml b/Cargo.toml index 218007a..ee80ed6 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -23,7 +23,13 @@ tempfile = "3" [features] libfuzzer = [] -klee = [] +# The `klee` feature (the KLEE runtime) lives in klee/Cargo.toml, the +# manifest of the rustc 1.72.1 bitcode build (D1, route A'). It is not a +# feature here: `--all-features` would link it with `libfuzzer` (both define +# `__VERIFIER_nondet_*`) and against KLEE intrinsics no host binary has. + +[lints.rust] +unexpected_cfgs = { level = "warn", check-cfg = ['cfg(feature, values("klee"))'] } [profile.release] opt-level = 3 diff --git a/rust/src/memtrack/mod.rs b/rust/src/memtrack/mod.rs index f47fcc5..6c4d15c 100644 --- a/rust/src/memtrack/mod.rs +++ b/rust/src/memtrack/mod.rs @@ -2,7 +2,16 @@ #![deny(unsafe_code)] use serde::Serialize; -use std::collections::HashSet; +use std::collections::{hash_map::DefaultHasher, HashSet}; +use std::hash::BuildHasherDefault; + +/// A `HashSet` with fixed SipHash keys instead of `RandomState`. +/// +/// `RandomState::new()` seeds itself from the OS (`getrandom`, reached +/// through a weak symbol), which under KLEE is an invalid function pointer +/// and kills the path (D1 route A'). These sets only answer membership +/// questions, so the hash seed changes nothing in their results. +pub(crate) type FixedHashSet = HashSet>; /// Single memory tracking record — analogous to `memtrack_log_t` in C. /// Addresses are `usize` (platform pointer width) instead of `long`. @@ -157,7 +166,9 @@ pub fn is_invalid_free_resolved(entries: &[MemTrackEntry], address: usize) -> bo .iter() .rev() .find(|e| (e.is_dynamic || e.is_free) && e.points_to == address) - .is_none_or(|e| e.is_free) + // `map_or(true, ..)`, not `is_none_or`: the KLEE runtime builds this + // file with rustc 1.72 (D1 condition 5). + .map_or(true, |e| e.is_free) } /// Checks whether dereferencing `address` is invalid (not allocated or already freed). @@ -177,7 +188,7 @@ pub fn is_deref_error(entries: &[MemTrackEntry], address: usize) -> bool { /// Checks whether there is a memory cleanup error (leaked allocation still referenced). /// Replaces `has_a_memcleanup_error_in_cntr`. pub fn has_memcleanup_error(entries: &[MemTrackEntry]) -> bool { - let mut resolved = HashSet::new(); + let mut resolved = FixedHashSet::default(); entries.iter().rev().any(|entry| { if !entry.is_dynamic && !entry.is_free { return false; @@ -355,7 +366,7 @@ impl DevelopLogs { .iter() .rev() .find(|r| r.var == pointer) - .is_none_or(|r| r.points_to == address) + .map_or(true, |r| r.points_to == address) }) } } @@ -409,7 +420,7 @@ pub fn is_invalid_address(entries: &[MemTrackEntry], address: usize, size: usize .checked_add(entry.size_destiny) .is_some_and(|allocation_end| address_end <= allocation_end) }; - let mut decided = HashSet::new(); + let mut decided = FixedHashSet::default(); !entries.iter().rev().any(|entry| { if entry.is_dynamic || entry.is_free { decided.insert(entry.points_to) && !entry.is_free && covers(entry) diff --git a/rust/src/output/legacy_files.rs b/rust/src/output/legacy_files.rs index 1131efd..153f3f7 100644 --- a/rust/src/output/legacy_files.rs +++ b/rust/src/output/legacy_files.rs @@ -161,7 +161,7 @@ pub fn write_checked_error(dir: &Path) -> io::Result<()> { /// `map2check_track_bb` keeps only the first block of each line /// (`Map2CheckFunctions.c:49-55`), so later rows of a line are skipped. pub fn write_trackbb_log(dir: &Path, rows: &[BasicBlockEntry]) -> io::Result<()> { - let mut seen = std::collections::HashSet::new(); + let mut seen = crate::memtrack::FixedHashSet::default(); let mut out = String::new(); for row in rows.iter().filter(|r| seen.insert(r.line)) { let _ = writeln!(out, "{};{}", row.line, row.function_name); diff --git a/rust/src/state.rs b/rust/src/state.rs index e564573..3529e22 100644 --- a/rust/src/state.rs +++ b/rust/src/state.rs @@ -1,7 +1,7 @@ // SPDX-License-Identifier: GPL-3.0-only #![deny(unsafe_code)] -use std::sync::{LazyLock, Mutex}; +use std::sync::{Mutex, OnceLock}; use crate::{ analysismode::mode::AnalysisMode, @@ -12,8 +12,9 @@ use crate::{ nondet::NonDetEntry, }; -static ANALYSIS_STATE: LazyLock> = - LazyLock::new(|| Mutex::new(AnalysisState::default())); +// `OnceLock` and not `LazyLock`: the KLEE runtime compiles this file with +// rustc 1.72 (D1 condition 5, klee/Cargo.toml), where `LazyLock` is unstable. +static ANALYSIS_STATE: OnceLock> = OnceLock::new(); static CONFIG: Mutex = Mutex::new(Config::new()); @@ -121,7 +122,7 @@ impl AnalysisState { } fn ensure_state() -> &'static Mutex { - &ANALYSIS_STATE + ANALYSIS_STATE.get_or_init(|| Mutex::new(AnalysisState::default())) } /// Starts a fresh analysis: forces the singleton into existence and runs the @@ -147,6 +148,10 @@ fn with_state_recovering(f: impl FnOnce(&mut AnalysisState)) { Err(poisoned) => { let mut guard = poisoned.into_inner(); f(&mut guard); + // `Mutex::clear_poison` is unstable in rustc 1.72, which builds + // the KLEE runtime. That build has `panic = "abort"`, so no + // caught panic can poison the lock there (D1, "Custo do pin"). + #[cfg(not(feature = "klee"))] state.clear_poison(); } } From 6ed460bd07296cfc714e40cac1b2070bc729305a Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:13:25 -0400 Subject: [PATCH 39/63] test: add the KLEE paths integration test (#20) tests/klee_paths.sh builds the symex runtime bitcode, links each program of tests/klee/ to it with llvm-link-16 and runs KLEE 3.1 with the frontend's flags, in docker (tests/klee/Dockerfile: the Map2Check dev image plus rustup 1.82.0 and 1.72.1): - prog.c: the D1 spike's program as the passes leave it; 2 completed paths and a ktest with x = 42; - reach.c, range.c, json.c: the witness nondet log and the JSON report must not fork KLEE (D1 condition 7); - deref.c: KLEE finds the off-by-one read, the property file says FALSE-DEREF; - layout.c: the error path's ktest, turned into a seed as the frontend's exportKleeVectorsAsSeeds does, replayed by the Rust libFuzzer reader, reaches the same error (D1 condition 9). With DEVELOP_SRC it also runs the programs on develop's C KLEE runtime and prints both path counts. It fails until the runtime exists. --- tests/klee/Dockerfile | 16 ++ tests/klee/deref.c | 24 +++ tests/klee/json.c | 18 ++ tests/klee/ktest_to_seed.py | 60 ++++++ tests/klee/layout.c | 25 +++ tests/klee/lower-intrinsics.ll | 23 +++ tests/klee/m2c.h | 127 +++++++++++++ tests/klee/prog.c | 16 ++ tests/klee/range.c | 16 ++ tests/klee/reach.c | 16 ++ tests/klee_paths.sh | 331 +++++++++++++++++++++++++++++++++ 11 files changed, 672 insertions(+) create mode 100644 tests/klee/Dockerfile create mode 100644 tests/klee/deref.c create mode 100644 tests/klee/json.c create mode 100755 tests/klee/ktest_to_seed.py create mode 100644 tests/klee/layout.c create mode 100644 tests/klee/lower-intrinsics.ll create mode 100644 tests/klee/m2c.h create mode 100644 tests/klee/prog.c create mode 100644 tests/klee/range.c create mode 100644 tests/klee/reach.c create mode 100755 tests/klee_paths.sh diff --git a/tests/klee/Dockerfile b/tests/klee/Dockerfile new file mode 100644 index 0000000..b0a4646 --- /dev/null +++ b/tests/klee/Dockerfile @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: GPL-3.0-only +# +# Image for tests/klee_paths.sh: Map2Check's development image (clang-16, +# LLVM 16 tools, KLEE 3.1, AFL++) plus rustup with the two toolchains the +# runtime needs: 1.82.0 (main build, the libFuzzer runtime of the cross +# test) and 1.72.1 (LLVM 16, the KLEE runtime, D1 route A'). Same recipe as +# the D1 spike's spikes/llvm16/Dockerfile.rust. +# +# docker build -t map2check-dev:cutover-klee-rust tests/klee +FROM map2check-dev:cutover-f01e8c3e +USER root +ENV RUSTUP_HOME=/opt/rustup CARGO_HOME=/opt/cargo PATH=/opt/cargo/bin:$PATH +RUN curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain 1.82.0 --profile minimal \ + && rustup toolchain install 1.72.1 --profile minimal \ + && chmod -R a+rwX /opt/rustup /opt/cargo +USER map2check diff --git a/tests/klee/deref.c b/tests/klee/deref.c new file mode 100644 index 0000000..b13751e --- /dev/null +++ b/tests/klee/deref.c @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Off-by-one read: i == 4 loads past the 4-int block. The memtrack check + * before the load must report it (FALSE-DEREF), and KLEE must find that + * path. */ +#include +#include "m2c.h" + +int __map2check_main__(void) { + M2C_START(M2C_MODE_MEMTRACK); + int *p = malloc(4 * sizeof(int)); + TRACK_MALLOC(p, (int)(4 * sizeof(int)), 11); + for (int k = 0; k < 4; k++) + p[k] = k; + int i = NONDET(int, int, 14); + int v = 0; + if (i >= 0 && i <= 4) { + CHECK_LOAD(&p[i], (int)sizeof(int), 17); + v = p[i]; + } + TRACK_FREE(p, 20); + free(p); + map2check_success(); + return v; +} diff --git a/tests/klee/json.c b/tests/klee/json.c new file mode 100644 index 0000000..f952721 --- /dev/null +++ b/tests/klee/json.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * D1 condition 7, the spike's case: with no analysis mode the runtime + * prints the JSON report at map2check_success, serde_json formatting an + * unconstrained x. The spike measured 22 paths instead of 2 for that; with + * the value concretised first there is one. Rust runtime only (develop has + * no JSON report). */ +#ifdef M2C_DEVELOP +#error "json.c exercises the Rust runtime's JSON report only" +#endif +#include "m2c.h" + +int __map2check_main__(void) { + map2check_init(); + int x = NONDET(int, int, 15); + (void)x; + map2check_success(); + return 0; +} diff --git a/tests/klee/ktest_to_seed.py b/tests/klee/ktest_to_seed.py new file mode 100755 index 0000000..a2b42fe --- /dev/null +++ b/tests/klee/ktest_to_seed.py @@ -0,0 +1,60 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-3.0-only +"""Turns a KLEE .ktest into an AFL++/libFuzzer seed, as Map2Check's frontend +does (develop: modules/frontend/test_suite/ktest_reader.cpp, readKtestFile +and ktestToFuzzerBytes, called by Caller::exportKleeVectorsAsSeeds): the +bytes of every object, in file order, concatenated. Nothing is reordered, +padded or byte-swapped, so the seed is exactly what the generators stored. + +Usage: ktest_to_seed.py +Prints one line per object: name, size and the bytes in hex.""" +import struct +import sys + + +def read_ktest(path): + with open(path, "rb") as f: + data = f.read() + pos = 0 + + def take(n): + nonlocal pos + if pos + n > len(data): + raise ValueError(f"{path}: truncated at byte {pos}") + chunk = data[pos : pos + n] + pos += n + return chunk + + def u32(): + return struct.unpack(">I", take(4))[0] + + magic = take(5) + if magic not in (b"KTEST", b"BOUT\n"): + raise ValueError(f"{path}: not a ktest file") + version = u32() + for _ in range(u32()): # arguments + take(u32()) + if version >= 2: + u32() # symArgvs + u32() # symArgvLen + objects = [] + for _ in range(u32()): + name = take(u32()).decode() + objects.append((name, take(u32()))) + return objects + + +def main(): + if len(sys.argv) != 3: + sys.exit(__doc__) + objects = read_ktest(sys.argv[1]) + if not objects: + sys.exit(f"{sys.argv[1]}: no objects") + for name, raw in objects: + print(f"{name} {len(raw)} {raw.hex()}") + with open(sys.argv[2], "wb") as f: + f.write(b"".join(raw for _, raw in objects)) + + +if __name__ == "__main__": + main() diff --git a/tests/klee/layout.c b/tests/klee/layout.c new file mode 100644 index 0000000..3a342f4 --- /dev/null +++ b/tests/klee/layout.c @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * D1 condition 9: the KLEE vector of this path, turned into a fuzzer seed + * the way the frontend does (exportKleeVectorsAsSeeds: the objects' bytes + * concatenated in order), must drive the AFL++/libFuzzer reader of the Rust + * runtime down the same path. One value per width and signedness, each a + * pattern no other byte layout would produce. */ +#include "m2c.h" + +int __map2check_main__(void) { + M2C_START(M2C_MODE_TARGET); + char c = NONDET(char, char, 11); + short s = NONDET(short, short, 12); + unsigned short us = NONDET(ushort, unsigned short, 13); + int x = NONDET(int, int, 14); + unsigned u = NONDET(uint, unsigned, 15); + long l = NONDET(long, long, 16); + unsigned long ul = NONDET(ulong, unsigned long, 17); + unsigned char uc = NONDET(uchar, unsigned char, 18); + if (c == 'A' && s == -2 && us == 0xBEEF && x == 42 && u == 0xDEADBEEFu && + l == -5 && ul == 0x0123456789ABCDEFul && uc == 0x7F) { + REACH_ERROR(21); + } + map2check_success(); + return 0; +} diff --git a/tests/klee/lower-intrinsics.ll b/tests/klee/lower-intrinsics.ll new file mode 100644 index 0000000..3a000ef --- /dev/null +++ b/tests/klee/lower-intrinsics.ll @@ -0,0 +1,23 @@ +; SPDX-License-Identifier: GPL-3.0-only +; tests/klee_paths.sh runs this with lli-16 before and after +; scripts/klee-lower-intrinsics.py: the outputs must be identical. +declare i64 @llvm.vector.reduce.add.v4i64(<4 x i64>) +declare i64 @llvm.vector.reduce.or.v2i64(<2 x i64>) +declare <8 x i16> @llvm.x86.sse2.pmulhu.w(<8 x i16>, <8 x i16>) +declare void @llvm.x86.sse2.pause() +declare i32 @printf(ptr, ...) +@f = private constant [29 x i8] c"%lld %lld %d %d %d %d %d %d\0A\00" +define i32 @main() { + %a = call i64 @llvm.vector.reduce.add.v4i64(<4 x i64> ) + %b = call i64 @llvm.vector.reduce.or.v2i64(<2 x i64> ) + %m = call <8 x i16> @llvm.x86.sse2.pmulhu.w(<8 x i16> , <8 x i16> ) + %m0 = extractelement <8 x i16> %m, i32 0 + %m1 = extractelement <8 x i16> %m, i32 1 + %m7 = extractelement <8 x i16> %m, i32 7 + %z0 = zext i16 %m0 to i32 + %z1 = zext i16 %m1 to i32 + %z7 = zext i16 %m7 to i32 + call void @llvm.x86.sse2.pause() + call i32 (ptr, ...) @printf(ptr @f, i64 %a, i64 %b, i32 %z0, i32 %z1, i32 %z7, i32 0, i32 0, i32 0) + ret i32 0 +} diff --git a/tests/klee/m2c.h b/tests/klee/m2c.h new file mode 100644 index 0000000..b80e880 --- /dev/null +++ b/tests/klee/m2c.h @@ -0,0 +1,127 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * + * The calls the Map2Check passes put into an analysed program, in two + * dialects: the Rust runtime's (default) and develop's C runtime's + * (-DM2C_DEVELOP), so tests/klee_paths.sh runs one program against both and + * compares the KLEE paths. `main` is already renamed to __map2check_main__, + * as the Map2CheckLibrary pass does; the runtime supplies `main`. + * + * Mode values: Rust `AnalysisMode` (rust/src/analysismode/mode.rs). */ +#ifndef M2C_TEST_H +#define M2C_TEST_H + +#include + +#define M2C_MODE_MEMTRACK 1 +#define M2C_MODE_TARGET 5 + +#ifdef M2C_DEVELOP +/* develop: Map2CheckFunctions.h, NonDetGenerator.h, NonDetLog.c, + * AnalysisModeMemtrack.c */ +void map2check_init(int isSvComp); +void map2check_success(void); +void map2check_target_function(const char *func_name, int scope, int line); +void map2check_malloc(void *ptr, int size); +void map2check_load(void *ptr, int size); +void map2check_check_deref(int line, const char *function_name); +void map2check_add_store_pointer(void *var, void *value, unsigned scope, + const char *name, int line, + const char *function_name); +void map2check_free(const char *name, void *ptr, unsigned scope, unsigned line, + const char *function_name); +#define M2C_DECL(t, ct) \ + ct map2check_non_det_##t(void); \ + void map2check_nondet_##t(unsigned line, unsigned scope, int value, \ + const char *function_name); +M2C_DECL(int, int) +M2C_DECL(char, char) +M2C_DECL(short, short) +M2C_DECL(ushort, unsigned short) +M2C_DECL(uchar, unsigned char) +M2C_DECL(long, long) +M2C_DECL(ulong, unsigned long) +M2C_DECL(uint, unsigned) +M2C_DECL(bool, int) +#define M2C_START(mode) map2check_init(0) +/* NonDetPass: the call is renamed to map2check_non_det_, and its value, + * cast to int, is logged with map2check_nondet_ right after. */ +#define NONDET(t, ct, line) \ + ({ \ + ct v_ = map2check_non_det_##t(); \ + map2check_nondet_##t(line, 0, (int)v_, "main"); \ + v_; \ + }) +#define REACH_ERROR(line) map2check_target_function("reach_error", 0, line) +#define TRACK_MALLOC(var, size, line) \ + do { \ + map2check_malloc((var), (size)); \ + map2check_add_store_pointer(&(var), (var), 0, #var, line, "main"); \ + } while (0) +#define CHECK_LOAD(addr, size, line) \ + do { \ + map2check_load((void *)(addr), (size)); \ + map2check_check_deref(line, "main"); \ + } while (0) +#define TRACK_FREE(var, line) map2check_free(#var, &(var), 0, line, "main") +#else +/* Rust runtime: include/map2check.h and docs/cutover/symbol-matrix.tsv */ +void map2check_set_mode(int mode); +void map2check_enable_witness(void); +void map2check_init(void); +void map2check_success(void); +void map2check_reach_target(const char *func, int scope, int line); +void map2check_map_malloc(const void *ptr_address, int size); +void map2check_map_store_pointer(const void *var_address, const void *value, + unsigned scope, const char *var_name, + int line_number, const char *funct_name); +void map2check_check_load(const void *ptr, int line, unsigned scope, int size, + const char *function_name); +void map2check_check_free(const char *name, const void *var_address, + const void *ptr, unsigned scope, unsigned line, + const char *function_name); +void map2check_map_free(const char *var_name, const void *var_address, + const void *ptr_address, unsigned scope, + unsigned line_number, const char *function_name); +#define M2C_DECL(t, ct) \ + ct __VERIFIER_nondet_##t(void); \ + void *map2check_save_nondet_log_##t(int line, int scope, int type_var, \ + ct *value, const char *function_name); +M2C_DECL(int, int) +M2C_DECL(char, char) +M2C_DECL(short, short) +M2C_DECL(ushort, unsigned short) +M2C_DECL(uchar, unsigned char) +M2C_DECL(long, long) +M2C_DECL(ulong, unsigned long) +M2C_DECL(uint, unsigned) +_Bool __VERIFIER_nondet_bool(void); +void *map2check_save_nondet_log_bool(int line, int scope, int type_var, + int *value, const char *function_name); +#define M2C_START(mode) \ + do { \ + map2check_set_mode(mode); \ + map2check_enable_witness(); \ + map2check_init(); \ + } while (0) +#define NONDET(t, ct, line) \ + ({ \ + ct v_ = __VERIFIER_nondet_##t(); \ + map2check_save_nondet_log_##t(line, 0, 0, &v_, "main"); \ + v_; \ + }) +#define REACH_ERROR(line) map2check_reach_target("reach_error", 0, line) +#define TRACK_MALLOC(var, size, line) \ + do { \ + map2check_map_malloc((var), (size)); \ + map2check_map_store_pointer(&(var), (var), 0, #var, line, "main"); \ + } while (0) +#define CHECK_LOAD(addr, size, line) \ + map2check_check_load((const void *)(addr), line, 0, (size), "main") +#define TRACK_FREE(var, line) \ + do { \ + map2check_check_free(#var, &(var), (var), 0, line, "main"); \ + map2check_map_free(#var, &(var), (var), 0, line, "main"); \ + } while (0) +#endif + +#endif diff --git a/tests/klee/prog.c b/tests/klee/prog.c new file mode 100644 index 0000000..ab14d81 --- /dev/null +++ b/tests/klee/prog.c @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * spikes/llvm16/prog.c (D1 spike) as the passes leave it: main renamed, mode + * and init first, the nondet logged, map2check_success before each return. + * KLEE must explore both sides of x == 42: two paths. */ +#include "m2c.h" + +int __map2check_main__(void) { + M2C_START(M2C_MODE_TARGET); + int x = NONDET(int, int, 9); + if (x == 42) { + map2check_success(); + return 1; + } + map2check_success(); + return 0; +} diff --git a/tests/klee/range.c b/tests/klee/range.c new file mode 100644 index 0000000..7eea44d --- /dev/null +++ b/tests/klee/range.c @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * D1 condition 7. On the error path x is only bounded (x > 1000), so the + * witness nondet log that the violation writes would format a value with + * many possible digits: unless the runtime concretises x first, every digit + * forks KLEE. Expected: two paths, the logged value above 1000. */ +#include "m2c.h" + +int __map2check_main__(void) { + M2C_START(M2C_MODE_TARGET); + int x = NONDET(int, int, 10); + if (x > 1000) { + REACH_ERROR(12); + } + map2check_success(); + return 0; +} diff --git a/tests/klee/reach.c b/tests/klee/reach.c new file mode 100644 index 0000000..1a2d901 --- /dev/null +++ b/tests/klee/reach.c @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * x == 42 reaches the error. The violation writes the witness nondet log + * (klee_log.csv), which formats x: D1 condition 7 requires the runtime to + * concretise x first, or the formatting forks KLEE (2 paths became 22 in + * the spike). Expected: two paths, one of them an abort with x = 42. */ +#include "m2c.h" + +int __map2check_main__(void) { + M2C_START(M2C_MODE_TARGET); + int x = NONDET(int, int, 10); + if (x == 42) { + REACH_ERROR(12); + } + map2check_success(); + return 0; +} diff --git a/tests/klee_paths.sh b/tests/klee_paths.sh new file mode 100755 index 0000000..ddc71dd --- /dev/null +++ b/tests/klee_paths.sh @@ -0,0 +1,331 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: GPL-3.0-only +# +# Integration test of the KLEE (`symex`) runtime, Task 2.7 / issue #20, +# D1 route A' (docs/cutover/decisoes.md, docs/klee-feasibility.md). +# +# Builds the runtime bitcode with scripts/build-klee-runtime.sh, links each +# program of tests/klee/ to it with llvm-link-16 and runs KLEE 3.1 with the +# flags of Map2Check's frontend (caller.cpp:815-830). Checks: +# +# prog.c both sides of x == 42 explored: 2 completed paths, one ktest +# with x = 42 (the brief's Step 1); +# reach.c x == 42 reaches the error: an abort whose ktest holds 42, the +# witness nondet log says 42, and still 2 paths (condition 7: +# the log is formatted from concretised values); +# range.c x > 1000 reaches the error: the log formats a value with many +# possible digits, still 2 paths (condition 7); +# json.c no mode: the JSON report formats an unconstrained x, 1 path +# (the spike's 2 -> 22 case, condition 7); +# deref.c KLEE finds the off-by-one read and the property file says +# FALSE-DEREF; +# layout.c condition 9: the ktest of the error path, turned into a seed +# as the frontend's exportKleeVectorsAsSeeds does, replayed +# through the Rust libFuzzer reader (--features libfuzzer, +# profile runtime), reaches the same error; a zero seed of the +# same size does not. +# +# With DEVELOP_SRC=, the same programs also +# run against develop's C KLEE runtime (built from +# modules/backend/library/lib), and the path counts of both are printed. +# +# Usage: tests/klee_paths.sh (runs in docker, image below) +# tests/klee_paths.sh --inside (already in the image) +# Environment: KLEE_TEST_IMAGE (default map2check-dev:cutover-klee-rust, +# built from tests/klee/Dockerfile when missing), +# DEVELOP_SRC, KLEE_RESULTS (file for the path-count table). +set -euo pipefail + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" + +if [ "${1:-}" != "--inside" ]; then + IMAGE="${KLEE_TEST_IMAGE:-map2check-dev:cutover-klee-rust}" + if ! docker image inspect "$IMAGE" >/dev/null 2>&1; then + docker build -t "$IMAGE" "$ROOT/tests/klee" + fi + args=(--rm -v "$ROOT":/src -w /src) + if [ -n "${DEVELOP_SRC:-}" ]; then + args+=(-v "$(cd "$DEVELOP_SRC" && pwd)":/develop:ro -e DEVELOP_SRC=/develop) + fi + if [ -n "${KLEE_RESULTS:-}" ]; then + args+=(-e KLEE_RESULTS=/src/"${KLEE_RESULTS#"$ROOT"/}") + fi + exec docker run "${args[@]}" "$IMAGE" /src/tests/klee_paths.sh --inside +fi + +T="$ROOT/tests/klee" +mkdir -p "$ROOT/target" +W="$(mktemp -d "$ROOT/target/klee-paths.XXXXXX")" +failures=0 +pass() { echo "PASS: $*"; } +fail() { + echo "FAIL: $*" >&2 + failures=$((failures + 1)) +} + +# Frontend flags (caller.cpp:815-830, z3 is the default --smt-solver); +# --max-time keeps a path blow-up from hanging the test. +KLEE_FLAGS=(--external-calls=all --optimize --use-cex-cache + --solver-backend=z3 --libc=uclibc --max-time=300s) + +echo "== scripts/klee-lower-intrinsics.py keeps the intrinsics' semantics" +python3 -I "$ROOT/scripts/klee-lower-intrinsics.py" "$T/lower-intrinsics.ll" "$W/lowered.ll" >/dev/null +if [ "$(lli-16 "$T/lower-intrinsics.ll")" = "$(lli-16 "$W/lowered.ll")" ] && + ! grep -q '@llvm\.\(vector\|x86\)' "$W/lowered.ll"; then + pass "lower-intrinsics.ll: same output from the intrinsics and their scalar versions" +else + fail "lower-intrinsics.ll: the scalar versions differ from the intrinsics" +fi + +echo "== building the KLEE runtime" +CARGO_TARGET_DIR="$ROOT/target/klee-docker" \ + "$ROOT/scripts/build-klee-runtime.sh" "$W/rust-rt.bc" + +# develop's C runtime as the frontend links it for symex (caller.cpp:560-640: +# the common files, ContainerRealloc, the mode's files, NonDetGeneratorKlee +# and, for the witness binary KLEE runs, WitnessGeneration). +develop_runtime() { # + local lib="$DEVELOP_SRC/modules/backend/library/lib" files f + files=(Map2CheckFunctions TrackBBLog NonDetLog PropertyGenerator + ContainerRealloc NonDetGeneratorKlee WitnessGeneration) + case "$1" in + target) files+=(AnalysisModeNone) ;; + memtrack) files+=(AnalysisModeMemtrack AllocationLog ListLog HeapLog) ;; + esac + mkdir -p "$W/dev" + local bcs=() + for f in "${files[@]}"; do + [ -f "$W/dev/$f.bc" ] || clang-16 -c -emit-llvm "$lib/$f.c" -o "$W/dev/$f.bc" 2>/dev/null + bcs+=("$W/dev/$f.bc") + done + llvm-link-16 "${bcs[@]}" -o "$2" +} + +# run_klee +# full: explores every path (the counts); +# frontend: stops at the first abort, as the frontend does for a property +# (--exit-on-error-type=Abort), which fixes the final property +# file. +# Leaves the run in $W/--/ (cwd of KLEE, so the +# runtime's files land there) and sets PATHS_COMPLETED, PATHS_PARTIAL. +run_klee() { + local prog="$1" rt="$2" kind="$3" dir="$W/$1-$2-$3" rtbc cflags=() mode=target + mkdir -p "$dir" + [ "$prog" = deref ] && mode=memtrack + if [ "$rt" = develop ]; then + cflags=(-DM2C_DEVELOP) + rtbc="$W/dev-$mode.bc" + [ -f "$rtbc" ] || develop_runtime "$mode" "$rtbc" + else + rtbc="$W/rust-rt.bc" + fi + clang-16 -g -emit-llvm -c "${cflags[@]}" -I"$T" "$T/$prog.c" -o "$dir/prog.bc" + llvm-link-16 "$dir/prog.bc" "$rtbc" -o "$dir/linked.bc" + local extra=() + [ "$kind" = frontend ] && extra=(--exit-on-error-type=Abort) + (cd "$dir" && klee "${KLEE_FLAGS[@]}" "${extra[@]}" --output-dir=klee-out linked.bc \ + >klee.log 2>&1) || true + PATHS_COMPLETED="$(sed -n 's/.*KLEE: done: completed paths = \([0-9]*\).*/\1/p' "$dir/klee.log")" + PATHS_PARTIAL="$(sed -n 's/.*KLEE: done: partially completed paths = \([0-9]*\).*/\1/p' "$dir/klee.log")" + PATHS_COMPLETED="${PATHS_COMPLETED:-?}" + PATHS_PARTIAL="${PATHS_PARTIAL:-0}" + if grep -q 'HaltTimer invoked' "$dir/klee.log"; then + fail "$prog/$rt/$kind: KLEE hit --max-time" + fi +} + +# Prints "name size hex" per object of a ktest, writes the seed to $2. +ktest_objects() { python3 -I "$T/ktest_to_seed.py" "$1" "$2"; } + +abort_ktest() { # : the ktest of the (single) abort, or nothing + local err + err="$(find "$1/klee-out" -name '*.abort.err' | sort | head -n 1)" + [ -n "$err" ] && echo "${err%.abort.err}.ktest" +} + +declare -A COUNT +count() { COUNT["$1/$2"]="$PATHS_COMPLETED+$PATHS_PARTIAL"; } + +# ── prog.c: two paths, x = 42 among the ktests ──────────────────────────────── +echo "== prog.c" +run_klee prog rust full +count prog rust +if [ "$PATHS_COMPLETED" = 2 ]; then + pass "prog.c: KLEE: done: completed paths = 2" +else + fail "prog.c: completed paths = $PATHS_COMPLETED, expected 2 (see $W/prog-rust-full/klee.log)" +fi +found42=0 +for k in "$W"/prog-rust-full/klee-out/*.ktest; do + if ktest_objects "$k" "$W/seed.tmp" | grep -qx 'non_det_int 4 2a000000'; then found42=1; fi +done +if [ "$found42" = 1 ]; then + pass "prog.c: a ktest has non_det_int = 42 (2a000000)" +else + fail "prog.c: no ktest with x = 42" +fi + +# ── reach.c: the error path, concretised witness log ────────────────────────── +echo "== reach.c" +run_klee reach rust full +count reach rust +total=$((PATHS_COMPLETED + PATHS_PARTIAL)) +if [ "$total" = 2 ]; then + pass "reach.c: 2 paths ($PATHS_COMPLETED completed + $PATHS_PARTIAL partially completed)" +else + fail "reach.c: $PATHS_COMPLETED completed + $PATHS_PARTIAL partial paths, expected 2 in all" +fi +run_klee reach rust frontend +d="$W/reach-rust-frontend" +k="$(abort_ktest "$d")" +if [ -n "$k" ] && ktest_objects "$k" "$W/seed.tmp" | grep -qx 'non_det_int 4 2a000000'; then + pass "reach.c: the abort's ktest has x = 42" +else + fail "reach.c: no abort with x = 42" +fi +if [ "$(head -n 1 "$d/map2check_property" 2>/dev/null)" = TARGET-REACHED ]; then + pass "reach.c: map2check_property says TARGET-REACHED" +else + fail "reach.c: map2check_property is '$(cat "$d/map2check_property" 2>/dev/null)'" +fi +# klee_log.csv: the value is the 6th field (see range.c below) +if [ "$(cut -d';' -f6 "$d/klee_log.csv" 2>/dev/null | head -n 1)" = 42 ]; then + pass "reach.c: klee_log.csv logs the concretised value 42" +else + fail "reach.c: klee_log.csv does not log 42: '$(cat "$d/klee_log.csv" 2>/dev/null)'" +fi + +# ── range.c, json.c: formatting a non-unique value (condition 7) ───────────── +echo "== range.c" +run_klee range rust full +count range rust +total=$((PATHS_COMPLETED + PATHS_PARTIAL)) +if [ "$total" = 2 ]; then + pass "range.c: 2 paths, the witness log of x > 1000 does not fork" +else + fail "range.c: $PATHS_COMPLETED completed + $PATHS_PARTIAL partial paths, expected 2 in all" +fi +run_klee range rust frontend +d="$W/range-rust-frontend" +# klee_log.csv row: step;line;scope;function;?;value;type, as develop's +# nondet_log_to_file writes it (same file from both runtimes). +logged="$(cut -d';' -f6 "$d/klee_log.csv" 2>/dev/null | head -n 1)" +if [[ "$logged" =~ ^[0-9]+$ ]] && [ "$logged" -gt 1000 ]; then + pass "range.c: klee_log.csv logs a concrete x = $logged > 1000" +else + fail "range.c: klee_log.csv value is '$logged'" +fi +echo "== json.c" +run_klee json rust full +count json rust +total=$((PATHS_COMPLETED + PATHS_PARTIAL)) +if [ "$total" = 1 ] && grep -q '"Container_NonDetLog"' "$W/json-rust-full/klee.log"; then + pass "json.c: the JSON report of an unconstrained x takes 1 path" +else + fail "json.c: $PATHS_COMPLETED completed + $PATHS_PARTIAL partial paths, expected 1 (spike: 22 unconcretised)" +fi + +# ── deref.c: FALSE-DEREF ────────────────────────────────────────────────────── +echo "== deref.c" +run_klee deref rust full +count deref rust +run_klee deref rust frontend +d="$W/deref-rust-frontend" +if [ -n "$(abort_ktest "$d")" ]; then + pass "deref.c: KLEE reports the violation as an abort" +else + fail "deref.c: no .abort.err ($(ls "$d/klee-out" | grep err || true))" +fi +if [ "$(head -n 1 "$d/map2check_property" 2>/dev/null)" = FALSE-DEREF ]; then + pass "deref.c: map2check_property says FALSE-DEREF" +else + fail "deref.c: map2check_property is '$(cat "$d/map2check_property" 2>/dev/null)'" +fi +k="$(abort_ktest "$d")" +if [ -n "$k" ] && ktest_objects "$k" "$W/seed.tmp" | grep -qx 'non_det_int 4 04000000'; then + pass "deref.c: the abort's ktest has i = 4" +else + fail "deref.c: the abort's ktest does not have i = 4" +fi + +# ── layout.c: ktest -> seed -> Rust libFuzzer reader (condition 9) ──────────── +echo "== layout.c" +run_klee layout rust full +count layout rust +run_klee layout rust frontend +d="$W/layout-rust-frontend" +k="$(abort_ktest "$d")" +expected_objects='non_det_char 1 41 +non_det_short 2 feff +non_det_ushort 2 efbe +non_det_int 4 2a000000 +non_det_uint 4 efbeadde +non_det_long 8 fbffffffffffffff +non_det_ulong 8 efcdab8967452301 +non_det_uchar 1 7f' +if [ -n "$k" ] && [ "$(ktest_objects "$k" "$W/layout.seed")" = "$expected_objects" ]; then + pass "layout.c: ktest objects are sizeof(type) little-endian bytes, in order, named non_det_" +else + fail "layout.c: ktest objects differ: $( [ -n "$k" ] && ktest_objects "$k" "$W/layout.seed" | tr '\n' ',')" +fi + +echo "== building the libFuzzer runtime (--features libfuzzer, profile runtime)" +MAIN_TARGET="$ROOT/target/klee-docker-main" +(cd "$ROOT" && CARGO_TARGET_DIR="$MAIN_TARGET" cargo build --locked --profile runtime \ + --features libfuzzer >"$W/cargo-main.log" 2>&1) || { + cat "$W/cargo-main.log" >&2 + fail "cargo build --profile runtime --features libfuzzer" +} +clang-16 -g -fsanitize=fuzzer -I"$T" "$T/layout.c" "$MAIN_TARGET/runtime/libmap2check.a" \ + -ldl -lpthread -lm -o "$W/layout.fuzz" +replay() { # : runs the seed as the frontend replays (argument) + mkdir -p "$2" + (cd "$2" && "$W/layout.fuzz" "$1" >replay.log 2>&1) && echo 0 || echo $? +} +if [ -s "$W/layout.seed" ]; then + rc="$(replay "$W/layout.seed" "$W/replay-klee")" + if [ "$rc" != 0 ] && [ "$(head -n 1 "$W/replay-klee/map2check_property" 2>/dev/null)" = TARGET-REACHED ]; then + pass "layout.c: the KLEE seed replayed by the libFuzzer reader reaches the error (rc=$rc, TARGET-REACHED)" + else + fail "layout.c: replay of the KLEE seed: rc=$rc, property '$(cat "$W/replay-klee/map2check_property" 2>/dev/null)'" + fi + head -c "$(stat -c %s "$W/layout.seed")" /dev/zero >"$W/zero.seed" + rc="$(replay "$W/zero.seed" "$W/replay-zero")" + if [ "$rc" = 0 ] && [ "$(head -n 1 "$W/replay-zero/map2check_property" 2>/dev/null)" = NONE ]; then + pass "layout.c: a zero seed of the same size does not reach it (control)" + else + fail "layout.c: zero seed: rc=$rc, property '$(cat "$W/replay-zero/map2check_property" 2>/dev/null)'" + fi +else + fail "layout.c: no seed from KLEE" +fi + +# ── develop's C runtime, same programs ──────────────────────────────────────── +if [ -n "${DEVELOP_SRC:-}" ]; then + echo "== develop C runtime ($DEVELOP_SRC)" + for prog in prog reach range deref layout; do + run_klee "$prog" develop full + count "$prog" develop + run_klee "$prog" develop frontend + echo "develop $prog: property '$(head -n 1 "$W/$prog-develop-frontend/map2check_property" 2>/dev/null)'," \ + "abort: $([ -n "$(abort_ktest "$W/$prog-develop-frontend")" ] && echo yes || echo no)" + done +fi + +table="$( + echo "| program | Rust runtime (completed+partial) | develop C runtime (completed+partial) |" + echo "|---|---|---|" + for prog in prog reach range json deref layout; do + echo "| $prog.c | ${COUNT[$prog/rust]:-?} | ${COUNT[$prog/develop]:-not run} |" + done +)" +echo "== KLEE paths (full exploration, $(klee --version 2>&1 | head -n 1))" +echo "$table" +[ -n "${KLEE_RESULTS:-}" ] && echo "$table" >"$KLEE_RESULTS" + +echo "work directory: $W" +if [ "$failures" -ne 0 ]; then + echo "$failures check(s) failed" >&2 + exit 1 +fi +echo "all KLEE checks passed" From 22c101ccda48ab1d74858d25adb0a71b7d840767 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:13:39 -0400 Subject: [PATCH 40/63] feat: add KLEE nondet runtime (#20) D1 route A': the map2check-library sources, built by rustc 1.72.1 as one LLVM 16 bitcode module that llvm-link-16 joins to the analysed program for KLEE 3.1. - klee/Cargo.toml: second manifest (own [workspace], build = false, no dev-dependencies, rust-version 1.72, feature klee, dependency caps, release profile with panic = abort, fat LTO, codegen-units = 1) and its own Cargo.lock v3 from the MSRV-aware resolver. - rust/src/nondet/klee.rs: __VERIFIER_nondet_ -> klee_make_symbolic with sizeof(type) bytes, objects named non_det_ as develop's NonDetGeneratorKlee.c names them; __VERIFIER_assume -> klee_assume; __VERIFIER_error/_assert as in the libFuzzer generator (shared body); main -> __map2check_main__; pchar capped at 4096 like the AFL++ reader. - Concretisation before output (D1 condition 7): prepare_for_output replaces nondet values and memory-record addresses/sizes with klee_get_value_* values, constrained with klee_assume, before the witness logs and the JSON report are formatted. No-op elsewhere. - scripts/build-klee-runtime.sh : the exact build, checks the rustc flags and LLVM 16, opt-16 --strip-debug (condition 6), lowers the vector/x86 intrinsics of the precompiled std that KLEE 3.1 cannot execute (scripts/klee-lower-intrinsics.py), reads the result back with llvm-dis-16 and checks the exported and imported symbols. - The klee module stays out of the C header (cbindgen:ignore); the export checklists cover klee.rs. tests/klee_paths.sh passes; path counts equal develop's C runtime on prog/reach/deref/layout and lower on range (1+1 vs 1+7). --- klee/Cargo.lock | 128 +++++++++++++ klee/Cargo.toml | 58 ++++++ rust/src/ffi.rs | 2 +- rust/src/nondet/klee.rs | 301 +++++++++++++++++++++++++++++++ rust/src/nondet/libfuzzer.rs | 18 +- rust/src/nondet/mod.rs | 30 ++- rust/src/output/mod.rs | 17 +- rust/src/output/termination.rs | 1 + scripts/build-klee-runtime.sh | 123 +++++++++++++ scripts/klee-lower-intrinsics.py | 121 +++++++++++++ tests/ffi_exports_checklist.rs | 28 +++ tests/ffi_guard_coverage.rs | 6 +- 12 files changed, 812 insertions(+), 21 deletions(-) create mode 100644 klee/Cargo.lock create mode 100644 klee/Cargo.toml create mode 100644 rust/src/nondet/klee.rs create mode 100755 scripts/build-klee-runtime.sh create mode 100644 scripts/klee-lower-intrinsics.py diff --git a/klee/Cargo.lock b/klee/Cargo.lock new file mode 100644 index 0000000..f235e1b --- /dev/null +++ b/klee/Cargo.lock @@ -0,0 +1,128 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 3 + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "map2check-klee-runtime" +version = "1.0.0-rc1" +dependencies = [ + "serde", + "serde_json", + "thiserror", +] + +[[package]] +name = "memchr" +version = "2.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "serde" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "serde_json" +version = "1.0.154" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7e9cc8b1b85264074fbcc02a88680c4096b1e47df8f739dceb03bf482f04bd6" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "syn" +version = "3.0.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d78c8dee4c7bf0e14673097256fed6142ce9d3b85a408189d07482442145823b" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "thiserror" +version = "2.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec86235f5fcc2a73650310756d2ac5b138a5780bbbdfae3eeccec992c435ba4f" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "unicode-ident" +version = "1.0.27" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2c754d6c33795a1c324727428e5a7dedb5b06195f9890bdbcba760d3e246563" + +[[package]] +name = "zmij" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" diff --git a/klee/Cargo.toml b/klee/Cargo.toml new file mode 100644 index 0000000..3634452 --- /dev/null +++ b/klee/Cargo.toml @@ -0,0 +1,58 @@ +# SPDX-License-Identifier: GPL-3.0-only +# +# Second manifest: the runtime of the Map2Check `symex` generator (D1, route +# A', docs/cutover/decisoes.md). It compiles the same sources as the main +# build (../rust/src) with rustc 1.72.1, whose LLVM 16 writes bitcode that +# llvm-link-16 and KLEE 3.1 read. Build it with scripts/build-klee-runtime.sh. +# +# Why a directory of its own and not a workspace member: a workspace shares +# one Cargo.lock, and the main one is v4, which cargo 1.72 cannot parse; the +# main manifest also pulls build.rs/cbindgen and mockall, whose dependency +# graph needs edition 2024. This manifest has its own `[workspace]`, its own +# lockfile (v3) and none of those. +# +# Rules for this manifest (D1 condition 5, costs (a)-(c)): +# - no build script (`build = false`): the C header comes from the main build; +# - no dev-dependencies; +# - dependency caps that compile on 1.72 (thiserror 2.0.21 needs 1.77); +# - Cargo.lock regenerated only with the MSRV-aware resolver: +# rustup toolchain install 1.86.0 --profile minimal +# CARGO_RESOLVER_INCOMPATIBLE_RUST_VERSIONS=fallback RUSTUP_TOOLCHAIN=1.86.0 \ +# cargo generate-lockfile --manifest-path klee/Cargo.toml +# sed -i 's/^version = 4/version = 3/' klee/Cargo.lock +[package] +name = "map2check-klee-runtime" +version = "1.0.0-rc1" +edition = "2021" +rust-version = "1.72" +build = false +publish = false +description = "Map2Check runtime for the KLEE (symex) generator: the map2check-library sources built as LLVM 16 bitcode" +license = "GPL-3.0-only" +authors = ["Herbert Rocha "] + +[lib] +name = "map2check" +crate-type = ["staticlib"] +path = "../rust/src/lib.rs" + +[dependencies] +serde = { version = ">=1.0, <=1.0.229", features = ["derive"] } +serde_json = ">=1.0, <=1.0.154" +thiserror = ">=2.0, <=2.0.20" + +[features] +klee = [] + +# panic = "abort": nothing catches a panic under KLEE, and the unwind +# machinery is code KLEE would have to execute. Fat LTO and one codegen unit +# put std into the one module KLEE loads (without LTO KLEE cannot resolve +# `__rust_no_alloc_shim_is_unstable`); D1 condition 6. +[profile.release] +opt-level = 3 +lto = "fat" +codegen-units = 1 +panic = "abort" +debug = false + +[workspace] diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index 3f173f7..5e3e5f2 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -125,7 +125,7 @@ fn violated_property_from_raw(raw: i32) -> Option { /// - Unwinding profiles (`dev`, `release`, `runtime`; `panic = "unwind"`): /// a panic is caught here with `catch_unwind`, a diagnostic is written to /// stderr and `default` is returned, so no unwind ever reaches C. -/// - `panic = "abort"` builds (the future KLEE build): `catch_unwind` cannot +/// - `panic = "abort"` builds (the KLEE runtime, `klee/Cargo.toml`): `catch_unwind` cannot /// intercept anything; a panic aborts the process immediately and there is /// no recovery. Exports therefore must not rely on panics for control flow. /// diff --git a/rust/src/nondet/klee.rs b/rust/src/nondet/klee.rs new file mode 100644 index 0000000..cb20557 --- /dev/null +++ b/rust/src/nondet/klee.rs @@ -0,0 +1,301 @@ +// SPDX-License-Identifier: GPL-3.0-only +// +// KLEE nondeterministic value generator: the `symex` runtime of Map2Check +// (feature `klee`, D1 route A', docs/cutover/decisoes.md). +// +// Built only from klee/Cargo.toml, with rustc 1.72.1 (LLVM 16), as bitcode +// that llvm-link-16 joins to the analysed program +// (scripts/build-klee-runtime.sh). Everything this file and the rest of the +// crate use must therefore compile on rustc 1.72 (D1 condition 5): no +// `unsafe extern`, no `LazyLock`, no std API newer than 1.72. +// +// Semantics are those of Map2Check develop's C generator +// (`modules/backend/library/lib/NonDetGeneratorKlee.c`): +// +// - each `__VERIFIER_nondet_` makes a fresh object of `sizeof(type)` +// bytes symbolic with `klee_make_symbolic`, named `non_det_` as +// develop names it (`"non_det_" #type`). The frontend reads the type back +// from that name (`ktest_reader.cpp`, `decodeKtestObject`), and turns a +// `.ktest` into a fuzzer seed by concatenating the objects' bytes +// (`ktestToFuzzerBytes`): `sizeof(type)` bytes per value, in order, in +// memory order (little-endian on x86-64), which is exactly what the +// AFL++/libFuzzer reader of `libfuzzer.rs` consumes (D1 condition 9); +// - `__VERIFIER_assume` is `klee_assume`; +// - `main` calls the analysed program's renamed `main` +// (`NonDetGeneratorKlee.c:16`); +// - values that reach the output (the witness logs, the JSON report) are +// concretised first with `klee_get_value_*` ([`concretize`], D1 +// condition 7): formatting a symbolic value forks KLEE on every digit. +// +// Violations end the state through the core's `std::process::abort()` +// (`output/termination.rs`), which KLEE reports as an abort error, as it +// does for develop's `abort()` in `map2check_exit`. +#![allow(unsafe_code)] + +use std::ffi::{c_char, c_int, c_void}; + +use super::NonDetValue; +use crate::state::AnalysisState; + +extern "C" { + fn klee_make_symbolic(addr: *mut c_void, nbytes: usize, name: *const c_char); + fn klee_assume(condition: usize); + fn klee_get_value_i32(expr: i32) -> i32; + fn klee_get_value_i64(expr: i64) -> i64; + /// The analysed program's `main`, renamed by the Map2CheckLibrary pass. + fn __map2check_main__(argc: c_int, argv: *mut *mut c_char) -> c_int; +} + +/// A fresh symbolic value of `T`'s size. `name` is NUL-terminated. +/// +/// `T` is one of the integer types below: any bit pattern is a valid `T`, +/// so KLEE may give the object any value. +fn symbolic(name: &'static [u8]) -> T { + debug_assert_eq!(name.last(), Some(&0)); + let mut value = T::default(); + // SAFETY: `value` is a live local of `size_of::()` bytes, and `name` + // is a NUL-terminated static string. + unsafe { + klee_make_symbolic( + (&mut value as *mut T).cast::(), + std::mem::size_of::(), + name.as_ptr().cast::(), + ); + } + value +} + +// ── Public nondet generators ───────────────────────────────────────────────── + +macro_rules! klee_nondet { + ($(#[$doc:meta])* $fn_name:ident, $ty:ty, $name:literal) => { + $(#[$doc])* + #[no_mangle] + pub extern "C" fn $fn_name() -> $ty { + crate::ffi::ffi_guard(|| symbolic::<$ty>(concat!($name, "\0").as_bytes()), 0) + } + }; +} + +klee_nondet!(__VERIFIER_nondet_char, i8, "non_det_char"); +klee_nondet!(__VERIFIER_nondet_uchar, u8, "non_det_uchar"); +klee_nondet!(__VERIFIER_nondet_short, i16, "non_det_short"); +klee_nondet!(__VERIFIER_nondet_ushort, u16, "non_det_ushort"); +klee_nondet!(__VERIFIER_nondet_int, i32, "non_det_int"); +klee_nondet!( + /// develop has `map2check_non_det_uint` and `_unsigned`; both are + /// `__VERIFIER_nondet_uint` in the core (symbol-matrix.tsv) and the + /// frontend reads both names as a 4-byte unsigned. + __VERIFIER_nondet_uint, + u32, + "non_det_uint" +); +klee_nondet!(__VERIFIER_nondet_long, i64, "non_det_long"); +klee_nondet!(__VERIFIER_nondet_ulong, u64, "non_det_ulong"); +klee_nondet!(__VERIFIER_nondet_size_t, usize, "non_det_size_t"); +klee_nondet!( + /// glibc's `loff_t` is `long long`. + __VERIFIER_nondet_loff_t, + i64, + "non_det_loff_t" +); +klee_nondet!( + /// develop: `#define sector_t unsigned long` (`NonDetGenerator.h:112`). + __VERIFIER_nondet_sector_t, + std::os::raw::c_ulong, + "non_det_sector_t" +); + +/// develop's `bool` is `typedef int bool` (`NonDetGenerator.h:74`): a 4-byte +/// object, returned as that `int` converted to `_Bool`, as `libfuzzer.rs` +/// reads it. +#[no_mangle] +pub extern "C" fn __VERIFIER_nondet_bool() -> bool { + crate::ffi::ffi_guard(|| symbolic::(b"non_det_bool\0") != 0, false) +} + +/// Four bytes, like `int`; develop has no `float` generator. The frontend +/// does not know the name `non_det_float` and decodes it as a signed int. +#[no_mangle] +pub extern "C" fn __VERIFIER_nondet_float() -> f32 { + crate::ffi::ffi_guard(|| f32::from_bits(symbolic::(b"non_det_float\0")), 0.0) +} + +#[no_mangle] +pub extern "C" fn __VERIFIER_nondet_double() -> f64 { + crate::ffi::ffi_guard(|| f64::from_bits(symbolic::(b"non_det_double\0")), 0.0) +} + +#[no_mangle] +pub extern "C" fn __VERIFIER_nondet_pointer() -> *mut c_void { + // An integer turned into a pointer; nothing here dereferences it. + crate::ffi::ffi_guard( + || symbolic::(b"non_det_pointer\0") as *mut c_void, + std::ptr::null_mut(), + ) +} + +/// `MAP2CHECK_MAX_FUZZED_STRING` of the AFL++ reader (`libfuzzer.rs`). +const MAX_FUZZED_STRING: u32 = 4096; + +/// A NUL-terminated string allocated with the C `malloc`, as develop's +/// `map2check_non_det_pchar` (`NonDetGeneratorKlee.c:40-55`): a symbolic +/// `unsigned` length (object `non_det_unsigned`), NULL when it is 0 or +/// when `malloc` fails, then `length - 1` symbolic chars (`non_det_char` +/// each) and a NUL. +/// +/// Unlike develop's KLEE generator, the length is capped at 4096 as the +/// AFL++ reader caps it, so that a KLEE vector replays through the fuzzer +/// reader as the same string (D1 condition 9). +#[no_mangle] +pub extern "C" fn __VERIFIER_nondet_pchar() -> *mut c_char { + extern "C" { + fn malloc(size: usize) -> *mut c_void; + } + crate::ffi::ffi_guard( + || { + let length = symbolic::(b"non_det_unsigned\0").min(MAX_FUZZED_STRING) as usize; + if length == 0 { + return std::ptr::null_mut(); + } + // SAFETY: plain C allocation; the result is checked for NULL. + let string = unsafe { malloc(length) }.cast::(); + if string.is_null() { + return std::ptr::null_mut(); + } + for i in 0..length - 1 { + // SAFETY: `i < length`, inside the `length`-byte block. + unsafe { string.add(i).write(symbolic::(b"non_det_char\0")) }; + } + // SAFETY: `length - 1 < length`. + unsafe { string.add(length - 1).write(0) }; + string.cast() + }, + std::ptr::null_mut(), + ) +} + +// ── SV-COMP verifier primitives ────────────────────────────────────────────── + +/// `nondet_assume` of develop's KLEE generator: `klee_assume(expr)`. A +/// false condition ends the path (KLEE stops exploring it); one that is +/// provably false is a KLEE user error, as in develop. +#[no_mangle] +pub extern "C" fn __VERIFIER_assume(condition: i32) { + crate::ffi::ffi_guard( + || { + // SAFETY: KLEE intrinsic, takes the condition by value. + unsafe { klee_assume(usize::from(condition != 0)) } + }, + (), + ); +} + +/// SV-COMP reachability primitive, as in `libfuzzer.rs`: records a +/// `Reachability` violation and aborts, which KLEE reports as an abort +/// error. +#[no_mangle] +pub extern "C" fn __VERIFIER_error() { + crate::ffi::ffi_guard(super::record_verifier_error, ()); + std::process::abort(); +} + +/// SV-COMP assert primitive: `__VERIFIER_error()` when `condition` is false. +#[no_mangle] +pub extern "C" fn __VERIFIER_assert(condition: i32) { + crate::ffi::ffi_guard( + || { + if condition == 0 { + __VERIFIER_error(); + } + }, + (), + ); +} + +/// KLEE's entry point. develop's KLEE generator defines `main` the same way +/// (`NonDetGeneratorKlee.c:16`): the passes renamed the program's `main` to +/// `__map2check_main__` and inserted the mode, init and success calls. +#[no_mangle] +pub extern "C" fn main(argc: c_int, argv: *mut *mut c_char) -> c_int { + crate::ffi::ffi_guard( + // SAFETY: the analysed program's entry point, linked by llvm-link. + || unsafe { __map2check_main__(argc, argv) }, + 0, + ) +} + +// ── Concretisation before output (D1 condition 7) ──────────────────────────── + +/// A concrete value of `expr` in the current path, which the path is then +/// constrained to. Constraining it keeps the values concretised one after +/// another jointly consistent (as KLEE does when it concretises the +/// arguments of an external call such as develop's `fprintf`). +fn concrete_i64(expr: i64) -> i64 { + // SAFETY: KLEE intrinsics, by value. + unsafe { + let value = klee_get_value_i64(expr); + klee_assume(usize::from(expr == value)); + value + } +} + +fn concrete_i32(expr: i32) -> i32 { + // SAFETY: KLEE intrinsics, by value. + unsafe { + let value = klee_get_value_i32(expr); + klee_assume(usize::from(expr == value)); + value + } +} + +fn concrete_usize(expr: usize) -> usize { + concrete_i64(expr as i64) as usize +} + +fn concrete_value(value: &NonDetValue) -> NonDetValue { + use NonDetValue as V; + match *value { + V::Int(v) => V::Int(concrete_i32(v)), + V::UInt(v) => V::UInt(concrete_i32(v as i32) as u32), + V::Long(v) => V::Long(concrete_i64(v)), + V::Char(v) => V::Char(concrete_i32(i32::from(v)) as i8), + V::Float(v) => V::Float(f32::from_bits(concrete_i32(v.to_bits() as i32) as u32)), + V::Double(v) => V::Double(f64::from_bits(concrete_i64(v.to_bits() as i64) as u64)), + V::Unsigned(v) => V::Unsigned(concrete_i32(v as i32) as u32), + V::Pointer(v) => V::Pointer(concrete_usize(v)), + V::UShort(v) => V::UShort(concrete_i32(i32::from(v)) as u16), + V::Short(v) => V::Short(concrete_i32(i32::from(v)) as i16), + V::ULong(v) => V::ULong(concrete_i64(v as i64) as u64), + V::Bool(v) => V::Bool(concrete_i32(v)), + V::UChar(v) => V::UChar(concrete_i32(i32::from(v)) as u8), + V::PChar(v) => V::PChar(concrete_usize(v)), + V::SizeT(v) => V::SizeT(concrete_usize(v)), + V::LoffT(v) => V::LoffT(concrete_i64(v)), + V::SectorT(v) => V::SectorT(concrete_i64(v as i64) as u64), + } +} + +fn concretize_memtrack(entry: &mut crate::memtrack::MemTrackEntry) { + entry.var_addr = concrete_usize(entry.var_addr); + entry.points_to = concrete_usize(entry.points_to); + entry.size_destiny = concrete_usize(entry.size_destiny); + entry.size_primitive = concrete_usize(entry.size_primitive); +} + +/// Replaces every value of the state that the output formats and that the +/// program can make symbolic (nondet values; addresses and sizes of the +/// memory records, which a symbolic size or index reaches) by a concrete +/// value of the current path. Called right before the witness logs or the +/// JSON report are written (`output::prepare_for_output`). +pub(crate) fn concretize(state: &mut AnalysisState) { + for entry in &mut state.nondets { + entry.value = concrete_value(&entry.value); + } + for entry in &mut state.memtrack { + concretize_memtrack(entry); + } + if let Some(entry) = state.failed_free.as_mut() { + concretize_memtrack(entry); + } +} diff --git a/rust/src/nondet/libfuzzer.rs b/rust/src/nondet/libfuzzer.rs index 8150cab..2d9127d 100644 --- a/rust/src/nondet/libfuzzer.rs +++ b/rust/src/nondet/libfuzzer.rs @@ -319,23 +319,7 @@ pub extern "C-unwind" fn map2check_fuzzer_assume(condition: i32) { /// being checked was violated; it never returns. #[no_mangle] pub extern "C" fn __VERIFIER_error() { - crate::ffi::ffi_guard( - || { - if let Err(e) = crate::state::with_state(|s| { - s.set_false( - crate::caller::ViolatedProperty::Reachability, - 0, - "__VERIFIER_error", - ); - Ok(()) - }) { - eprintln!("[map2check] error: {e}"); - } - // Out of the lock, and only without a mode (#19 ruling). - crate::output::print_report_unless_mode_set(); - }, - (), - ); + crate::ffi::ffi_guard(super::record_verifier_error, ()); std::process::abort(); } diff --git a/rust/src/nondet/mod.rs b/rust/src/nondet/mod.rs index c21bfee..c4743cf 100644 --- a/rust/src/nondet/mod.rs +++ b/rust/src/nondet/mod.rs @@ -1,5 +1,5 @@ // SPDX-License-Identifier: GPL-3.0-only -#![cfg_attr(not(feature = "libfuzzer"), deny(unsafe_code))] +#![cfg_attr(not(any(feature = "libfuzzer", feature = "klee")), deny(unsafe_code))] use serde::Serialize; @@ -100,6 +100,34 @@ impl NonDetEntry { #[cfg(feature = "libfuzzer")] pub mod libfuzzer; +/// KLEE nondeterministic value generator (`symex`) — feature-gated, built +/// only from `klee/Cargo.toml` with rustc 1.72.1 (D1, route A'). Not in the +/// C header: the KLEE runtime is linked as bitcode, and its `main` and the +/// KLEE intrinsics it imports are not library API. +/// +/// cbindgen:ignore +#[cfg(feature = "klee")] +pub mod klee; + +/// Body of `__VERIFIER_error`, the same for every generator: records a +/// `Reachability` violation and, without a mode, prints the JSON report. +/// The export aborts afterwards. +#[cfg(any(feature = "libfuzzer", feature = "klee"))] +fn record_verifier_error() { + if let Err(e) = crate::state::with_state(|s| { + s.set_false( + crate::caller::ViolatedProperty::Reachability, + 0, + "__VERIFIER_error", + ); + Ok(()) + }) { + eprintln!("[map2check] error: {e}"); + } + // Out of the lock, and only without a mode (#19 ruling). + crate::output::print_report_unless_mode_set(); +} + #[cfg(test)] mod tests { use super::*; diff --git a/rust/src/output/mod.rs b/rust/src/output/mod.rs index 3476f74..c42572c 100644 --- a/rust/src/output/mod.rs +++ b/rust/src/output/mod.rs @@ -79,6 +79,18 @@ pub fn to_json(state: &AnalysisState) -> Result { serde_json::to_string_pretty(&log).map_err(Map2CheckError::from) } +/// Makes the state safe to format, right before a report or a log file is +/// written from it. Under KLEE (feature `klee`) values the program made +/// symbolic are replaced by concrete values of the current path, so that +/// formatting them does not fork KLEE (D1 condition 7); elsewhere it does +/// nothing. +pub(crate) fn prepare_for_output(state: &mut AnalysisState) { + #[cfg(feature = "klee")] + crate::nondet::klee::concretize(state); + #[cfg(not(feature = "klee"))] + let _ = state; +} + /// Print the JSON report to stdout, as the C library did. A failed write /// (closed stdout, EPIPE) is reported on stderr instead of panicking. pub fn print_json(state: &AnalysisState) -> Result<(), Map2CheckError> { @@ -102,7 +114,10 @@ pub fn print_report_unless_mode_set() { if crate::state::config().mode.is_some() { return; } - match crate::state::with_state(|s| to_json(s)) { + match crate::state::with_state(|s| { + prepare_for_output(s); + to_json(s) + }) { Ok(json) => write_report(&json), Err(e) => eprintln!("[map2check] error: {e}"), } diff --git a/rust/src/output/termination.rs b/rust/src/output/termination.rs index b52db37..17830b9 100644 --- a/rust/src/output/termination.rs +++ b/rust/src/output/termination.rs @@ -97,6 +97,7 @@ fn write_aux_files(dir: &Path, config: Config, s: &AnalysisState, end_of_program fn exit(dir: &Path, config: Config, end_of_program: bool) { let failed = match state::with_state(|s| { if config.witness { + super::prepare_for_output(s); write_aux_files(dir, config, s, end_of_program); } mode::analysis_destroy(config.mode, s); diff --git a/scripts/build-klee-runtime.sh b/scripts/build-klee-runtime.sh new file mode 100755 index 0000000..6b956a7 --- /dev/null +++ b/scripts/build-klee-runtime.sh @@ -0,0 +1,123 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: GPL-3.0-only +# +# Builds the runtime of the Map2Check `symex` generator as one LLVM 16 +# bitcode module (D1, route A', docs/cutover/decisoes.md): +# +# rustc 1.72.1 (LLVM 16) on klee/Cargo.toml with its own Cargo.lock, +# feature `klee`, panic = abort, fat LTO, codegen-units = 1, +# --emit=llvm-bc, then `opt-16 --strip-debug` (condition 6), then the +# vector intrinsics KLEE cannot run lowered to scalar code +# (scripts/klee-lower-intrinsics.py), then a read-back with `llvm-dis-16`. +# +# Usage: scripts/build-klee-runtime.sh +# +# Environment: +# KLEE_RUSTC_TOOLCHAIN rustup toolchain (default 1.72.1) +# OPT, LLVM_DIS, LLVM_NM, LLVM_AS LLVM 16 tools (default opt-16, +# llvm-dis-16, llvm-nm-16, llvm-as-16) +# CARGO_TARGET_DIR target directory (default /target/klee) +# +# Every step must succeed; the script stops at the first failure. +set -euo pipefail + +die() { + echo "build-klee-runtime: error: $*" >&2 + exit 1 +} + +[ $# -eq 1 ] || die "usage: $0 " +out="$1" +case "$out" in + *.bc) ;; + *) die "the output must end in .bc: $out" ;; +esac + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" +TOOLCHAIN="${KLEE_RUSTC_TOOLCHAIN:-1.72.1}" +OPT="${OPT:-opt-16}" +LLVM_DIS="${LLVM_DIS:-llvm-dis-16}" +LLVM_NM="${LLVM_NM:-llvm-nm-16}" +LLVM_AS="${LLVM_AS:-llvm-as-16}" +TARGET_DIR="${CARGO_TARGET_DIR:-$ROOT/target/klee}" + +for tool in cargo rustc python3 "$OPT" "$LLVM_DIS" "$LLVM_NM" "$LLVM_AS"; do + command -v "$tool" >/dev/null || die "$tool not found in PATH" +done + +# The bitcode must come from LLVM 16: a newer LLVM writes attributes that +# llvm-link-16 and KLEE 3.1 reject ("Unknown attribute kind (91)"). +llvm_version="$(RUSTUP_TOOLCHAIN="$TOOLCHAIN" rustc -vV | sed -n 's/^LLVM version: //p')" +case "$llvm_version" in + 16.*) ;; + *) die "rustc $TOOLCHAIN uses LLVM '$llvm_version', not 16" ;; +esac + +deps="$TARGET_DIR/release/deps" +# Always rebuild the crate itself (not its dependencies): the rustc command +# line checked below only shows up when it is compiled, and a stale module +# would be picked up. +RUSTUP_TOOLCHAIN="$TOOLCHAIN" CARGO_TARGET_DIR="$TARGET_DIR" \ + cargo clean --release -p map2check-klee-runtime \ + --manifest-path "$ROOT/klee/Cargo.toml" >/dev/null 2>&1 || die "cargo clean failed" +if [ -d "$deps" ]; then + find "$deps" -maxdepth 1 -name 'map2check-*.bc' -delete +fi + +log="$(mktemp)" +trap 'rm -f "$log" "$log.ll" "$log.lowered.ll"' EXIT +# --locked: the dependency graph is the one in klee/Cargo.lock (v3), never +# a fresh resolution. The profile (klee/Cargo.toml) sets panic = abort, +# lto = fat and codegen-units = 1; the check below proves rustc got them. +if ! RUSTUP_TOOLCHAIN="$TOOLCHAIN" CARGO_TARGET_DIR="$TARGET_DIR" \ + cargo rustc --locked --release --features klee --lib -v \ + --manifest-path "$ROOT/klee/Cargo.toml" -- --emit=llvm-bc >"$log" 2>&1; then + cat "$log" >&2 + die "cargo rustc failed" +fi + +crate_line="$(grep -E 'Running `.*rustc .*--crate-name map2check ' "$log" | tail -n 1 || true)" +[ -n "$crate_line" ] || { cat "$log" >&2; die "no rustc command line for the map2check crate"; } +for flag in '-C panic=abort' '-C lto' '-C codegen-units=1' '--emit=llvm-bc'; do + case "$crate_line" in + *"$flag"*) ;; + *) die "rustc was not given '$flag': $crate_line" ;; + esac +done +case "$crate_line" in + *'-C lto=thin'* | *'-C lto=off'* | *'-C lto=no'*) die "LTO is not fat: $crate_line" ;; +esac + +mapfile -t modules < <(find "$deps" -maxdepth 1 -name 'map2check-*.bc') +[ "${#modules[@]}" -eq 1 ] || die "expected one map2check-*.bc in $deps, found ${#modules[@]}" + +# The debug info of the precompiled std does not survive llvm-link-16 + +# KLEE ("Broken module found"); only the runtime loses it, the analysed +# program keeps its -g. +mkdir -p "$(dirname "$out")" +"$OPT" --strip-debug "${modules[0]}" -o "$out" || die "$OPT --strip-debug failed" +"$LLVM_DIS" "$out" -o /dev/null || die "$LLVM_DIS cannot read $out" + +# KLEE 3.1 cannot execute the vector/x86 intrinsics of the precompiled std +# ("unimplemented intrinsic" in core::fmt padding): replace them by scalar +# functions. Fails on an intrinsic the script does not know. +"$LLVM_DIS" "$out" -o "$log.ll" || die "$LLVM_DIS cannot read $out" +python3 -I "$ROOT/scripts/klee-lower-intrinsics.py" "$log.ll" "$log.lowered.ll" || + die "lowering the vector intrinsics failed" +"$LLVM_AS" "$log.lowered.ll" -o "$out" || die "$LLVM_AS rejected the lowered module" +rm -f "$log.ll" "$log.lowered.ll" +"$LLVM_DIS" "$out" -o /dev/null || die "$LLVM_DIS cannot read $out" + +# The module must define what the symex pipeline links against, and leave +# the KLEE intrinsics to KLEE. +defined="$("$LLVM_NM" --defined-only "$out")" +for sym in main __VERIFIER_nondet_int __VERIFIER_nondet_double __VERIFIER_assume \ + __VERIFIER_error map2check_init map2check_set_mode map2check_success; do + grep -Eq "[[:space:]]$sym\$" <<<"$defined" || die "$out does not define $sym" +done +undefined="$("$LLVM_NM" --undefined-only "$out")" +for sym in klee_make_symbolic klee_assume __map2check_main__; do + grep -Eq "[[:space:]]$sym\$" <<<"$undefined" || die "$out does not import $sym" +done + +echo "build-klee-runtime: wrote $out (rustc $TOOLCHAIN, LLVM $llvm_version)" diff --git a/scripts/klee-lower-intrinsics.py b/scripts/klee-lower-intrinsics.py new file mode 100644 index 0000000..26061f9 --- /dev/null +++ b/scripts/klee-lower-intrinsics.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-3.0-only +"""Replaces the vector and x86 intrinsics of the KLEE runtime's IR by plain +scalar functions, which KLEE 3.1 can execute. + +The precompiled std of rustc 1.72.1 is vectorised (char counting in +`core::fmt::Formatter::pad`/`pad_integral`, `StrSearcher::new`, integer +formatting), and fat LTO carries that code into the runtime module. KLEE 3.1 +does not implement `llvm.vector.reduce.*` nor the x86 SIMD intrinsics: a path +that formats a padded number (the witness logs of the memory modes do) dies +with "unimplemented intrinsic". Each declaration found is renamed to an +internal function with the same signature and semantics, written with +extractelement/insertelement and scalar arithmetic, which KLEE supports. + +Any other `llvm.vector.*` or `llvm.x86.*` declaration is an error: a new +rustc brought one this script does not know, and it must be added here. + +`llvm.bitreverse.*` (miniz_oxide, which only std's backtrace printing +reaches) is left alone: KLEE warns "unsupported intrinsic" at load, and its +own --optimize would fold a scalar version back into the intrinsic anyway. + +Usage: klee-lower-intrinsics.py """ +import re +import sys + +REDUCE = re.compile( + r"^declare\s+(?Pi\d+)\s+@(?Pllvm\.vector\.reduce\." + r"(?Padd|mul|and|or|xor|umax|umin|smax|smin)\.v(?P\d+)i(?P\d+))" + r"\(<(?P=n) x i(?P=w)>\)[^\n]*$", + re.M, +) +PMULHU = re.compile(r"^declare\s+<8 x i16>\s+@(?Pllvm\.x86\.sse2\.pmulhu\.w)\([^\n]*$", re.M) +PAUSE = re.compile(r"^declare\s+void\s+@(?Pllvm\.x86\.sse2\.pause)\(\)[^\n]*$", re.M) +LEFTOVER = re.compile(r"^declare[^\n]*@(llvm\.(?:vector|x86)\.[\w.]+)", re.M) + + +def new_name(name): + return "__map2check_klee_" + name.replace(".", "_") + + +def reduce_body(name, op, n, w): + t = f"i{w}" + lines = [f"define internal {t} @{new_name(name)}(<{n} x {t}> %v) {{"] + for i in range(n): + lines.append(f" %e{i} = extractelement <{n} x {t}> %v, i32 {i}") + acc = "%e0" + for i in range(1, n): + r = f"%r{i}" + if op in ("add", "mul", "and", "or", "xor"): + lines.append(f" {r} = {op} {t} {acc}, %e{i}") + else: + pred = {"umax": "ugt", "umin": "ult", "smax": "sgt", "smin": "slt"}[op] + lines.append(f" %c{i} = icmp {pred} {t} {acc}, %e{i}") + lines.append(f" {r} = select i1 %c{i}, {t} {acc}, {t} %e{i}") + acc = r + lines.append(f" ret {t} {acc}") + lines.append("}") + return "\n".join(lines) + + +def pmulhu_body(name): + lines = [f"define internal <8 x i16> @{new_name(name)}(<8 x i16> %a, <8 x i16> %b) {{"] + prev = "undef" + for i in range(8): + lines += [ + f" %a{i} = extractelement <8 x i16> %a, i32 {i}", + f" %b{i} = extractelement <8 x i16> %b, i32 {i}", + f" %xa{i} = zext i16 %a{i} to i32", + f" %xb{i} = zext i16 %b{i} to i32", + f" %m{i} = mul i32 %xa{i}, %xb{i}", + f" %h{i} = lshr i32 %m{i}, 16", + f" %t{i} = trunc i32 %h{i} to i16", + f" %v{i} = insertelement <8 x i16> {prev}, i16 %t{i}, i32 {i}", + ] + prev = f"%v{i}" + lines += [f" ret <8 x i16> {prev}", "}"] + return "\n".join(lines) + + +def pause_body(name): + return f"define internal void @{new_name(name)}() {{\n ret void\n}}" + + +def main(): + if len(sys.argv) != 3: + sys.exit(__doc__) + with open(sys.argv[1]) as f: + ir = f.read() + + bodies = [] + renamed = [] + for m in REDUCE.finditer(ir): + bodies.append(reduce_body(m["name"], m["op"], int(m["n"]), int(m["w"]))) + renamed.append(m["name"]) + for m in PMULHU.finditer(ir): + bodies.append(pmulhu_body(m["name"])) + renamed.append(m["name"]) + for m in PAUSE.finditer(ir): + bodies.append(pause_body(m["name"])) + renamed.append(m["name"]) + + for pattern in (REDUCE, PMULHU, PAUSE): + ir = pattern.sub("", ir) + for name in renamed: + ir = re.sub(r"@" + re.escape(name) + r"(?![\w.])", "@" + new_name(name), ir) + + leftover = sorted(set(LEFTOVER.findall(ir))) + if leftover: + sys.exit("klee-lower-intrinsics: no scalar version of: " + ", ".join(leftover)) + + with open(sys.argv[2], "w") as f: + f.write(ir) + f.write("\n; Scalar versions of vector/x86 intrinsics (scripts/klee-lower-intrinsics.py)\n") + f.write("\n\n".join(bodies)) + f.write("\n") + for name in renamed: + print(f"klee-lower-intrinsics: {name} -> {new_name(name)}") + + +if __name__ == "__main__": + main() diff --git a/tests/ffi_exports_checklist.rs b/tests/ffi_exports_checklist.rs index 024977f..f3f463e 100644 --- a/tests/ffi_exports_checklist.rs +++ b/tests/ffi_exports_checklist.rs @@ -302,10 +302,16 @@ plain!(not_an_export); ); } +/// O runtime KLEE (`rust/src/nondet/klee.rs`, feature `klee` do +/// `klee/Cargo.toml`) fica fora do header (`cbindgen:ignore`): é ligado como +/// bitcode, não pelo header. +const KLEE_RUNTIME: &str = "rust/src/nondet/klee.rs"; + #[test] fn todo_export_do_codigo_esta_no_header() { let rust_src: String = rust_sources() .iter() + .filter(|p| p.as_str() != KLEE_RUNTIME) .map(|p| read(p)) .collect::>() .join("\n"); @@ -319,6 +325,28 @@ fn todo_export_do_codigo_esta_no_header() { ); } +/// O runtime KLEE exporta os mesmos símbolos, com as mesmas assinaturas, que +/// o gerador libFuzzer declara no header, mais o `main` que o KLEE executa +/// (como `NonDetGeneratorKlee.c` de `develop`). +#[test] +fn exports_do_runtime_klee_sao_os_do_header_mais_main() { + let klee = source_exports(&read(KLEE_RUNTIME)); + let header = common::header_declarations(&read("include/map2check.h")); + let fora: Vec<_> = klee + .iter() + .filter(|name| name.as_str() != "main" && !header.contains_key(name.as_str())) + .collect(); + assert!(fora.is_empty(), "exports KLEE fora do header: {fora:?}"); + for name in [ + "main", + "__VERIFIER_nondet_int", + "__VERIFIER_assume", + "__VERIFIER_error", + ] { + assert!(klee.contains(name), "o runtime KLEE não exporta {name}"); + } +} + /// Arquivos `.rs` de `rust/src/`, relativos à raiz do crate. fn rust_sources() -> Vec { fn walk(dir: &Path, root: &Path, out: &mut Vec) { diff --git a/tests/ffi_guard_coverage.rs b/tests/ffi_guard_coverage.rs index ab22411..4194353 100644 --- a/tests/ffi_guard_coverage.rs +++ b/tests/ffi_guard_coverage.rs @@ -7,7 +7,11 @@ mod common; use common::read; -const SOURCES: &[&str] = &["rust/src/ffi.rs", "rust/src/nondet/libfuzzer.rs"]; +const SOURCES: &[&str] = &[ + "rust/src/ffi.rs", + "rust/src/nondet/libfuzzer.rs", + "rust/src/nondet/klee.rs", +]; /// Drops comments and string/char-literal contents so braces inside them do /// not confuse the body matcher. From 6d0d35fe75e310b947e8ce2fb39a0291cd47db6b Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:13:49 -0400 Subject: [PATCH 41/63] ci: build the KLEE runtime bitcode with rustc 1.72.1 New job klee_runtime on ubuntu-24.04: installs LLVM 16 (apt llvm-16) and rustc 1.72.1, runs scripts/build-klee-runtime.sh against klee/Cargo.toml with its own lockfile (--locked), checks the bitcode exists and that llvm-dis-16 reads it, runs cargo-deny on klee/Cargo.lock and uploads the .bc. It is the guard of D1 condition 5: clippy on 1.82 does not see std API newer than 1.72. KLEE itself does not run here (its image is Map2Check's dev image); tests/klee_paths.sh runs locally and in Map2Check's CI (Task 3.4). --- .github/workflows/CI.yml | 41 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 40 insertions(+), 1 deletion(-) diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index cef1d0d..263937f 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -156,4 +156,43 @@ jobs: run: cargo audit - name: Build do Dockerfile - run: docker build . \ No newline at end of file + run: docker build . + # Runtime do gerador `symex` (D1, rota A′; docs/klee-feasibility.md): as + # mesmas fontes compiladas por rustc 1.72.1 (LLVM 16) a partir de + # klee/Cargo.toml e do seu Cargo.lock v3, em bitcode que o llvm-link-16 e o + # KLEE 3.1 leem. O clippy em 1.82 do job acima não vê o que o 1.72 não + # compila (`LazyLock`, `unsafe extern`, API de std posterior a 1.72): este + # job é a guarda da condição 5. O KLEE em si não roda aqui (a imagem com o + # KLEE 3.1 é a do Map2Check); tests/klee_paths.sh roda no CI do Map2Check + # (Task 3.4) e localmente com Docker. + klee_runtime: + runs-on: ubuntu-24.04 + steps: + - name: Baixar o código do repositório + uses: actions/checkout@v4 + + - name: Instalar as ferramentas do LLVM 16 (opt, llvm-dis, llvm-nm, llvm-as) + run: sudo apt-get update && sudo apt-get install -y llvm-16 + + - name: Instalar o rustc 1.72.1 (LLVM 16) + run: rustup toolchain install 1.72.1 --profile minimal + + - name: Compilar o runtime KLEE em bitcode (klee/Cargo.toml, --locked) + run: scripts/build-klee-runtime.sh target/klee/map2check-klee.bc + + - name: Verificar que o bitcode existe e é legível pelo llvm-dis-16 + run: | + test -s target/klee/map2check-klee.bc + llvm-dis-16 target/klee/map2check-klee.bc -o /dev/null + + - name: Checar dependências do lockfile KLEE (cargo-deny) + uses: EmbarkStudios/cargo-deny-action@v2 + with: + manifest-path: ./klee/Cargo.toml + + - name: Publicar o bitcode do runtime KLEE + uses: actions/upload-artifact@v4 + with: + name: map2check-klee-bc + path: target/klee/map2check-klee.bc + if-no-files-found: error From 49eed878cb079551cf7a41518186f554d79630e1 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:13:57 -0400 Subject: [PATCH 42/63] docs: record the KLEE feasibility decision (go) and close D1 conditions 5, 6, 7 and 9 docs/klee-feasibility.md: the product (second manifest, lockfile, feature klee, build script, test, CI), the generator's semantics against develop's NonDetGeneratorKlee.c, the evidence of tests/klee_paths.sh (path counts Rust vs develop C), the findings (vector intrinsics of the precompiled std, RandomState under KLEE) and the limitations. decisoes.md: conditions 5, 6, 7 and 9 (KLEE side) marked ATENDIDA with their evidence. TODO.md: the KLEE gate item is done. --- TODO.md | 18 +++---- docs/cutover/decisoes.md | 10 +++- docs/klee-feasibility.md | 105 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 122 insertions(+), 11 deletions(-) create mode 100644 docs/klee-feasibility.md diff --git a/TODO.md b/TODO.md index ad3fccc..b9b2510 100644 --- a/TODO.md +++ b/TODO.md @@ -82,9 +82,9 @@ e não repetem as tarefas já listadas. [state.rs](rust/src/state.rs#L24) e [libfuzzer.rs](rust/src/nondet/libfuzzer.rs#L21). -- [ ] **Validar os fluxos de build e testes C/KLEE/LibFuzzer.** A feature Cargo - `klee` não tem implementação específica (`Cargo.toml` a declara, mas não há - uso correspondente em `rust/src`); o Dockerfile tolera falhas ao emitir/linkar +- [ ] **Validar os fluxos de build e testes C/KLEE/LibFuzzer.** A feature + `klee` está implementada (Task 2.7, [docs/klee-feasibility.md](docs/klee-feasibility.md)); + o Dockerfile tolera falhas ao emitir/linkar bitcode (`|| true`), e os scripts de integração ainda pressupõem o layout C em `build/`. Decidir e registrar se KLEE terá suporte Rust ou uma contingência C; em ambos os casos, fazer os builds falharem quando o artefato esperado @@ -135,13 +135,11 @@ e não repetem as tarefas já listadas. executem no CI; remover artefatos de build legados que não sejam mais usados. Falhas na geração ou no link de bitcode não devem ser mascaradas por `|| true`. -- [ ] **Executar o gate de viabilidade de KLEE e registrar a decisão.** - Prototipar a feature `klee` com nondeterminismo via `klee_make_symbolic`, - estado compatível com execução simbólica e emissão/link de bitcode; testar o - programa KLEE de referência com uma versão LLVM compatível. Registrar - evidências, limitações e a decisão go/no-go em - `docs/klee-feasibility.md`; se Rust não for viável, formalizar a contingência - de manter a biblioteca C nesse fluxo. +- [x] **Executar o gate de viabilidade de KLEE e registrar a decisão.** + Feito na Task 2.7 (#20): feature `klee` em `klee/Cargo.toml` (rustc 1.72.1, + bitcode LLVM 16, `scripts/build-klee-runtime.sh`), teste + `tests/klee_paths.sh` com KLEE 3.1, decisão **go** em + [docs/klee-feasibility.md](docs/klee-feasibility.md). - [ ] **Medir e controlar a regressão de desempenho.** Criar benchmarks reproduzíveis contra a biblioteca C, usando programas de referência e corpus diff --git a/docs/cutover/decisoes.md b/docs/cutover/decisoes.md index 4239bb0..303872a 100644 --- a/docs/cutover/decisoes.md +++ b/docs/cutover/decisoes.md @@ -68,7 +68,7 @@ klee --external-calls=all --optimize --use-cex-cache --libc=uclibc linked5.bc O frontend passa ainda `--solver-backend`, uma política de parada e de busca e a flag de sementes (`caller.cpp:815-822`). Essas flags não foram usadas no spike. -**A Task 2.7 precisa transformar isso em produto.** Peças concretas: +**A Task 2.7 precisa transformar isso em produto.** Peças concretas (feitas em 2026-10-16, ver `docs/klee-feasibility.md`; o build ganhou uma sexta peça, a troca dos intrínsecos vetoriais): 1. **Segundo manifesto** para o build KLEE, como membro de workspace ou diretório próprio (por exemplo `klee/Cargo.toml`). Ele aponta para as mesmas fontes de `rust/src`, com `build = false`, sem dev-dependencies e com `rust-version = "1.72"`. 2. **`Cargo.lock` v3 próprio** para esse manifesto, gerado com o comando acima e versionado. Ele é separado do lockfile v4 do build principal. @@ -138,9 +138,17 @@ A aprovação aceita a proposta junto com estas condições. **Mitigação (decisão do mantenedor, 2026-10-16, “Mitigar em Rust”):** o frontend dá ao AFL++ uma semente inicial de **pelo menos 8 bytes**, o tamanho do maior valor lido (`long`, `double`, ponteiros). Com isso o input-to-state tem o operando inteiro na entrada desde a primeira execução, sem depender de o corpus crescer por arestas do runtime. Responsável: Task 3.3, onde o frontend cria o diretório `-i` do afl-fuzz (`caller.cpp` ~893-905 em `develop`), com teste em `CallerLink` de que o arquivo de semente tem ≥ 8 bytes. Remedido no gate 4.2, no corpus real. 5. **MSRV 1.72 no que o build `symex` compila,** com os custos (a)–(c) acima e o `clear_poison` sob `#[cfg(not(feature = "klee"))]`. Responsável: Task 2.7, com o job de CI em 1.72.1. + + **Status: ATENDIDA (Task 2.7, 2026-10-16).** `klee/Cargo.toml` (diretório próprio com `[workspace]` vazio, `build = false`, sem dev-dependencies, `rust-version = "1.72"`) e `klee/Cargo.lock` v3 (serde 1.0.229, serde_json 1.0.154, thiserror 2.0.20) compilam `rust/src` com rustc 1.72.1 e `--locked`. Ajustes de fonte: `LazyLock` → `OnceLock`, `is_none_or` → `map_or(true, ..)` e `clear_poison` sob `#[cfg(not(feature = "klee"))]`; o build normal mantém o `clear_poison`, e o teste `reset_recovers_from_poisoned_lock` continua passando. O job `klee_runtime` do CI compila em 1.72.1 e roda `cargo-deny` no segundo lockfile. Evidência em `docs/klee-feasibility.md`. 6. **LTO fat e `--strip-debug` do runtime `symex` são obrigatórios.** O `-g` do programa analisado é preservado. Responsável: Task 2.7. + + **Status: ATENDIDA (Task 2.7, 2026-10-16).** `scripts/build-klee-runtime.sh` exige, na linha de comando do rustc, `-C panic=abort`, `-C lto` (fat), `-C codegen-units=1` e `--emit=llvm-bc`, além de LLVM 16. Em seguida roda `opt-16 --strip-debug` e lê o resultado de volta com `llvm-dis-16`. Os programas de `tests/klee_paths.sh` são compilados com `-g`. Achado: o KLEE 3.1 não executa os intrínsecos vetoriais do `std` pré-compilado (`llvm.vector.reduce.*`, `llvm.x86.sse2.pmulhu.w`), e o build passou a trocá-los por funções escalares (`scripts/klee-lower-intrinsics.py`). 7. **Concretizar antes de serializar no `symex`.** O `serde_json` formatando valores simbólicos levou o `prog.c` de 2 para 22 caminhos. O gerador KLEE do Rust precisa concretizar (`klee_get_value_*`) o que vai para o log e para o relatório, como o C faz ao escrever por chamada externa. Responsável: Task 2.7. + + **Status: ATENDIDA (Task 2.7, 2026-10-16).** `output::prepare_for_output` chama `nondet::klee::concretize` (`klee_get_value_i32/i64` seguido de `klee_assume(v == valor)`) antes dos logs de witness e do relatório JSON. Medido em `tests/klee_paths.sh`: `json.c` (relatório JSON de `x` sem restrição) dá 1 caminho, contra 21 sem a concretização; `range.c` (log de witness com `x > 1000`) dá 1 + 1, contra 1 + 7 sem ela e 1 + 7 no runtime C de `develop`, que bifurca dentro do `vfprintf` da uClibc. Correção da premissa: o C de `develop` não concretiza por chamada externa, porque o `fprintf` é o da uClibc, executado pelo KLEE. 8. **A rota B sob KLEE não é usada.** Além de não explorar símbolos, o KLEE 3.1 não tem `--load` (`Unknown command line argument '--load=...'`), e a `.so` só carrega com `LD_PRELOAD` e `-Wl,-z,lazy`. 9. **O layout do vetor tem de ser o mesmo entre KLEE e AFL++.** No modo padrão (`none` → AFL++ → KLEE → AFL++ com `--seed-exchange`), os vetores do KLEE viram sementes do AFL++ (`Caller::exportKleeVectorsAsSeeds`). O gerador AFL do Rust tem de consumir a semente exatamente como o KLEE a produz: `sizeof(tipo)` bytes por valor, em ordem, little-endian. É o que `read_u32` faz hoje para `int`, mas falta verificar os demais tipos. A semântica de entrada curta precisa estar alinhada com a do C (condição 4). Responsável: Tasks 2.5 e 2.7, com teste cruzado ktest → semente → replay. + + **Status, lado KLEE: ATENDIDA (Task 2.7, 2026-10-16).** O gerador KLEE do Rust nomeia os objetos `non_det_`, como `develop`, com `sizeof(tipo)` bytes. Em `layout.c` (`char`, `short`, `ushort`, `int`, `uint`, `long`, `ulong`, `uchar`), o ktest do caminho de erro tem os oito objetos em ordem e little-endian. Convertido em semente como `ktestToFuzzerBytes` faz (30 bytes concatenados) e rodado como argumento pelo leitor libFuzzer do Rust (`--features libfuzzer`, perfil `runtime`), ele chega ao mesmo erro (`TARGET-REACHED`). Uma semente de zeros do mesmo tamanho não chega. O `pchar` do KLEE passou a limitar o comprimento a 4096, como o leitor AFL++, para que o replay gere a mesma string. 10. **O rustc 1.72.1 (2023) fica preso ao LLVM 16.** Quando o Map2Check migrar de LLVM, o toolchain do `symex` precisa acompanhar ou ser removido pela rota C. 11. **O programa analisado é compilado com `-fexceptions`.** Responsável: Task 3.2. Um `__VERIFIER_assume` falso dentro de uma iteração do fuzzer desenrola a pilha (`resume_unwind`) através dos frames C do programa até o `LLVMFuzzerTestOneInput`. Sem `-fexceptions`, o clang marca toda função C como `nounwind`, e desenrolar a partir de uma função `nounwind` é comportamento indefinido no IR do LLVM. Hoje funciona só porque o x86-64 Linux emite `.eh_frame` por padrão. Com `-fno-asynchronous-unwind-tables -fno-unwind-tables` o processo aborta (“failed to initiate panic, error 5”), e o AFL++ contaria isso como crash: um falso positivo. O passo que gera o bitcode do programa (`clang -emit-llvm`) tem de passar `-fexceptions`, porque os atributos viajam no bitcode que o `afl-clang-fast` recebe. Nenhuma etapa pode acrescentar `-fno-unwind-tables` ou `-fno-asynchronous-unwind-tables`. Unwind tables sozinhas (`-funwind-tables`) não bastam: fazem o desenrolar funcionar, mas o IR continua `nounwind`. Evidência: `tests/ffi_harness.c` em `-O2` com clang-16 e afl-clang-fast (`scripts/cutover/d1-remeasure/results/flags16.txt`, tabela em `docs/api-deltas.md`, “`__VERIFIER_assume` under a fuzzer”). O CI roda o harness em `-O2` com e sem `-fexceptions`. diff --git a/docs/klee-feasibility.md b/docs/klee-feasibility.md new file mode 100644 index 0000000..14be3b8 --- /dev/null +++ b/docs/klee-feasibility.md @@ -0,0 +1,105 @@ +# Viabilidade do KLEE com o runtime Rust (issue #20, Task 2.7) + +Data: 2026-10-16. Decisão de origem: D1, rota A′ (`docs/cutover/decisoes.md`), com a evidência do spike em `docs/cutover/spike-llvm16.md`. + +**Decisão: GO.** O gerador `symex` do Map2Check usa o runtime Rust: as fontes de `rust/src`, compiladas por rustc 1.72.1 (LLVM 16.0.5) como um módulo de bitcode, ligado ao programa analisado com `llvm-link-16` e executado no KLEE 3.1. Nos seis programas de `tests/klee/`, o runtime Rust explora o mesmo número de caminhos que o runtime C de `develop`, ou menos, e chega aos mesmos veredictos. O vetor do KLEE, convertido em semente como o frontend faz, leva o leitor AFL++/libFuzzer do Rust ao mesmo ramo. + +## O produto + +| Peça | Arquivo | +|---|---| +| Segundo manifesto, com `build = false`, sem dev-dependencies, `rust-version = "1.72"`, `[features] klee = []`, tetos de dependência e `[workspace]` próprio | `klee/Cargo.toml` | +| Lockfile v3 próprio, gerado pelo resolvedor com fallback de MSRV (comando no cabeçalho do manifesto): serde 1.0.229, serde_json 1.0.154, thiserror 2.0.20 | `klee/Cargo.lock` | +| Gerador KLEE: `__VERIFIER_nondet_` → `klee_make_symbolic`, `__VERIFIER_assume` → `klee_assume`, `main` → `__map2check_main__`, concretização antes da saída | `rust/src/nondet/klee.rs` | +| Build exato: rustc 1.72.1, `--emit=llvm-bc`, `panic = "abort"`, LTO fat, `codegen-units = 1`, `opt-16 --strip-debug`, troca dos intrínsecos vetoriais, leitura de volta com `llvm-dis-16` | `scripts/build-klee-runtime.sh`, `scripts/klee-lower-intrinsics.py` | +| Teste de integração em Docker | `tests/klee_paths.sh`, `tests/klee/` (programas, `Dockerfile`, conversor ktest → semente) | +| Job de CI em rustc 1.72.1 | `.github/workflows/CI.yml`, job `klee_runtime` | + +**Por que um diretório próprio, e não um membro de workspace.** Um workspace compartilha um único `Cargo.lock`. O lockfile do build principal é v4, que o cargo 1.72 não lê. O manifesto principal também traz `build.rs`/cbindgen e `mockall`, cujo grafo exige edition 2024. Um membro de workspace herdaria os dois problemas. `klee/Cargo.toml` declara `[workspace]` vazio e é a raiz do próprio grafo. + +**A feature `klee` saiu do manifesto principal.** Ela era um marcador vazio. Mantida lá, `cargo test --all-features` ligaria `libfuzzer.rs` e `klee.rs` juntos: os dois definem `__VERIFIER_nondet_*`, e os intrínsecos do KLEE não existem num binário nativo. O manifesto principal declara o valor em `[lints.rust] unexpected_cfgs`, para que `#[cfg(feature = "klee")]` passe no clippy em 1.82. O módulo `klee` fica fora do header (`cbindgen:ignore`), e `include/map2check.h` sai byte a byte igual ao de antes. + +### Como construir e ligar + +```bash +scripts/build-klee-runtime.sh out/map2check-klee.bc # precisa de rustup 1.72.1 e das ferramentas LLVM 16 +clang-16 -g -emit-llvm -c programa-instrumentado.c -o prog.bc +llvm-link-16 prog.bc out/map2check-klee.bc -o linked.bc +klee --external-calls=all --optimize --use-cex-cache --solver-backend=z3 --libc=uclibc linked.bc +``` + +O programa já vem dos passes com o `main` renomeado para `__map2check_main__`. O `main` que o KLEE executa é o do runtime, como em `NonDetGeneratorKlee.c:16`. A primeira chamada do programa é `map2check_set_mode(m)`, depois `map2check_init()`. Para o binário de witness, o frontend também chama `map2check_enable_witness()`. + +## Semântica do gerador + +| Aspecto | `develop` (`NonDetGeneratorKlee.c`) | Rust (`nondet/klee.rs`) | +|---|---|---| +| Valor | `klee_make_symbolic(&v, sizeof v, "non_det_" #type)` | o mesmo, objeto de `size_of::()` bytes | +| Nome do objeto | `non_det_` | `non_det_`. `__VERIFIER_nondet_uint` cobre `map2check_non_det_uint` e `_unsigned` (`symbol-matrix.tsv`) e se chama `non_det_uint`; o frontend lê os dois nomes como unsigned de 4 bytes | +| `bool` | `int`, 4 bytes | 4 bytes (`non_det_bool`), devolvido como `int != 0`, igual ao leitor AFL++ | +| `float` | não existe | 4 bytes, `non_det_float`. O frontend não conhece o nome e o decodifica como int com sinal | +| `pchar` | comprimento `non_det_unsigned` sem limite, depois `length - 1` × `non_det_char` | o mesmo, **com teto de 4096** no comprimento, como o leitor AFL++, para que o vetor do KLEE gere a mesma string no replay (condição 9) | +| assume | `klee_assume(expr)` | `klee_assume(cond != 0)`. Um assume comprovadamente falso é erro de usuário do KLEE nos dois | +| `__VERIFIER_error` / `_assert` | `__VERIFIER_error` vazio. O alvo vira `map2check_target_function` | registra `Reachability` e aborta, como no gerador libFuzzer (corpo compartilhado em `nondet/mod.rs`) | +| Violação | `map2check_exit` → `abort()` | `output/termination.rs` → `std::process::abort()` → `abort` da uClibc. O KLEE registra `.abort.err` nos dois | +| Saída | `fprintf` da uClibc com o valor simbólico | `output::prepare_for_output` → `klee::concretize` antes de formatar (condição 7) | + +**Concretização.** Antes de escrever os logs de witness ou o relatório JSON, cada valor nondet e cada endereço ou tamanho dos registros de memória é trocado por `klee_get_value_i32/i64(v)`. Logo depois vem `klee_assume(v == valor)`. O assume torna os valores conjuntamente consistentes: o segundo valor concretizado respeita o primeiro, como acontece quando o KLEE concretiza os argumentos de uma chamada externa. Valores `float`/`double` são concretizados pelos bits, porque o KLEE não tem aritmética de ponto flutuante simbólica. + +## Evidência + +Os comandos rodaram na imagem `map2check-dev:cutover-klee-rust`, construída de `tests/klee/Dockerfile` (`map2check-dev:cutover-f01e8c3e` + rustup 1.82.0 e 1.72.1). A referência C é `Map2Check-cutover`, HEAD `b3c10cb7`. + +```bash +DEVELOP_SRC=../Map2Check-cutover tests/klee_paths.sh +``` + +Resultado: 16 PASS, 0 FAIL (`all KLEE checks passed`). + +| Programa | O que verifica | Rust: completos + parciais | `develop` C: completos + parciais | +|---|---|---|---| +| `prog.c` | o `prog.c` do spike, do jeito que os passes o deixam: os dois lados de `x == 42` | **2 + 0** (`completed paths = 2`; ktest com `non_det_int 4 2a000000`) | 2 + 0 | +| `reach.c` | `x == 42` atinge o erro: abort com ktest 42, `TARGET-REACHED`, `klee_log.csv` com 42 | 1 + 1 | 1 + 1 | +| `range.c` | `x > 1000` atinge o erro. O log formata um valor com muitos dígitos possíveis | **1 + 1** | **1 + 7** | +| `json.c` | sem modo, o relatório JSON de um `x` sem restrição (o caso 2 → 22 do spike) | 1 + 0 | não se aplica (o C não tem JSON) | +| `deref.c` | leitura fora do bloco com `i == 4`: abort, `FALSE-DEREF`, ktest com `i = 4` | 3 + 1 | 3 + 1 | +| `layout.c` | oito tipos. O ktest do erro vira semente e passa pelo leitor libFuzzer do Rust | 8 + 1 | 8 + 1 | + +"Parciais" são os caminhos encerrados por erro, aqui o abort da violação. Exploração completa, sem `--exit-on-error-type`. Os veredictos foram conferidos numa segunda execução com `--exit-on-error-type=Abort`, como o frontend roda os modos de propriedade. + +**Explosão de caminhos frente ao `develop`: nenhuma.** O único desvio vai no sentido contrário. Em `range.c`, o C formata o valor simbólico dentro do `vfprintf` da uClibc, e cada dígito bifurca. Lá aparecem 7 caminhos de erro, com um só `.abort.err` gravado, porque o KLEE ignora os erros repetidos no mesmo ponto. O Rust concretiza antes e fica com 1. O total de instruções também é menor no Rust em `deref.c` (163 908 contra 704 223) e em `layout.c` (138 383 contra 471 846). Em `prog.c` é maior (43 655 contra 28 049), pela inicialização do `std`. + +**Condição 7, prova pela falha.** Com `prepare_for_output` desligado (experimento, não versionado), `range.c` deu 1 + 7 caminhos e `json.c` deu 21 + 0. Com a concretização, 1 + 1 e 1 + 0. + +**Condição 9, teste cruzado.** O ktest do caminho de erro de `layout.c` traz os objetos abaixo, nesta ordem (`tests/klee/ktest_to_seed.py`, que reproduz `readKtestFile` + `ktestToFuzzerBytes`): + +``` +non_det_char 1 41 +non_det_short 2 feff +non_det_ushort 2 efbe +non_det_int 4 2a000000 +non_det_uint 4 efbeadde +non_det_long 8 fbffffffffffffff +non_det_ulong 8 efcdab8967452301 +non_det_uchar 1 7f +``` + +A semente tem esses 30 bytes concatenados. O mesmo `layout.c` foi ligado a `libmap2check.a` (`cargo build --profile runtime --features libfuzzer`, rustc 1.82) com `clang-16 -fsanitize=fuzzer` e rodado com a semente como argumento. Ele chegou ao erro: `rc=77` e `map2check_property` com `TARGET-REACHED`. Como controle, uma semente de 30 zeros termina com `rc=0` e `NONE`. + +**Condições 5 e 6 no build.** `scripts/build-klee-runtime.sh` confere, na linha de comando do rustc para o crate, que estão presentes `-C panic=abort`, `-C lto`, `-C codegen-units=1` e `--emit=llvm-bc`, e que a versão do LLVM é 16. Depois passa o módulo por `opt-16 --strip-debug` e lê o resultado de volta com `llvm-dis-16`. Também confere que o módulo define `main`, `__VERIFIER_nondet_*`, `__VERIFIER_assume`, `map2check_init`, `map2check_set_mode` e `map2check_success`, e que importa `klee_make_symbolic`, `klee_assume` e `__map2check_main__`. Os mesmos passos rodaram num `ubuntu:24.04` limpo, com `apt-get install llvm-16` e `rustup toolchain install 1.72.1`, que é o que o job de CI faz. O bitcode saiu com 7,5 MB. + +## Achados e limitações + +1. **Intrínsecos vetoriais do `std` pré-compilado.** O `std` do 1.72.1 vem vetorizado, e o LTO fat leva esse código para o módulo. O KLEE 3.1 não executa `llvm.vector.reduce.add/or` nem `llvm.x86.sse2.pmulhu.w`. O caminho de `deref.c` morria com `unimplemented intrinsic` em `core::fmt::Formatter::pad_integral`, ao formatar o log de heap em hexadecimal com largura. Desligar a vetorização no LTO (`-C no-vectorize-loops -C no-vectorize-slp`) não resolve, porque o código já vem vetorizado no bitcode do `std`. `scripts/klee-lower-intrinsics.py` troca cada intrínseco por uma função interna escalar (`extractelement`/`insertelement` e aritmética). A equivalência é conferida com `lli-16` em `tests/klee/lower-intrinsics.ll`. Um intrínseco vetorial ou x86 desconhecido faz o build falhar. Isto é uma sexta peça do produto, além das cinco da D1. +2. **`RandomState` do `HashSet`.** O `RandomState::new()` busca a semente do SO por um símbolo fraco (`getrandom`). Sob o KLEE, isso vira `invalid function pointer`, e o caminho morre ao gravar os logs de witness. Os três `HashSet` do crate passaram a usar `memtrack::FixedHashSet`, com chaves SipHash fixas. Eles só respondem pertinência, então o resultado não muda em nenhum build. +3. **`llvm.bitreverse` (miniz_oxide).** Só a impressão de backtrace do `std` chega a ele, e com `panic = "abort"` ela não é executada. O KLEE avisa `unsupported intrinsic` no carregamento. Não dá para trocá-lo: o `--optimize` do KLEE refaz o intrínseco a partir da versão escalar. +4. **Chamadas externas.** O `std` grava os arquivos por `open64`/`write`/`close`/`unlink` externos, como o `fopen`/`fprintf` do C. O KLEE avisa, por exemplo, `undefined reference to function: statx`, `pthread_key_create` e `_Unwind_*`. Nenhum desses símbolos é executado nos caminhos testados. +5. **`--external-calls=all` e o `std`.** O que usa símbolos fracos via `dlsym` ou `syscall!` (entropia, `statx`, TLS com destrutor) é inseguro sob o KLEE. Hoje o crate não usa nada disso no estado, mas qualquer código novo que entre no build `symex` precisa passar por `tests/klee_paths.sh`. +6. **Escopo dos testes.** Os programas imitam a instrumentação dos passes à mão (`tests/klee/m2c.h`). O pipeline real com os passes do Map2Check é a Task 3.x, e o `tests/klee_paths.sh` no CI do Map2Check é a Task 3.4. Overflow, memcleanup, `pchar` e `double` simbólicos não têm programa próprio. `layout.c` cobre os oito tipos inteiros. +7. **CI.** O job `klee_runtime` compila o bitcode em 1.72.1 com `--locked`, confere que ele existe e que o `llvm-dis-16` o lê, roda o `cargo-deny` sobre `klee/Cargo.lock` e publica o `.bc` como artefato. O KLEE não roda no CI desta biblioteca, porque a imagem com o KLEE 3.1 é a de desenvolvimento do Map2Check e não está num registro público. `tests/klee_paths.sh` roda localmente e entra no CI do Map2Check na Task 3.4. + +## Custos aceitos (D1, "Custo do pin do rustc 1.72.1") + +- O que o build `symex` compila (todo o `rust/src` fora dos testes) fica preso ao rustc 1.72. Os ajustes desta task foram: `LazyLock` → `OnceLock` em `state.rs`; `Option::is_none_or` (1.82) → `map_or(true, ..)` em `memtrack/mod.rs`; e `Mutex::clear_poison` sob `#[cfg(not(feature = "klee"))]`, que continua ativo no build normal. O spike também trocou `unsafe extern` por `extern`. Aqui não precisou, porque o único `unsafe extern` fica em `libfuzzer.rs`, que o build `symex` não compila. O job `klee_runtime` falha se algo posterior a 1.72 entrar. +- Dois lockfiles. `cargo deny check` roda no principal (job `build_and_test`) e no de `klee/` (job `klee_runtime`). +- Saída do pin: a rota C (LLVM 19 + KLEE 3.3), nos gatilhos listados na D1. From 2721a5f8ded1cbda73581ec6aafaeac934e4c72f Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:33:01 -0400 Subject: [PATCH 43/63] fix: make __VERIFIER_error/__VERIFIER_assert no-ops with a mode set develop's __VERIFIER_error (Map2CheckFunctions.c:57-59) and __VERIFIER_assert (AnalysisModeAssert.c:25-27) do nothing; the passes report through map2check_target_function and map2check_assert. Both Rust generators recorded REACHABILITY and aborted in every mode, with no property file. Under KLEE's --exit-on-error-type=Abort that ended the exploration before the real violation of a memory or overflow task, and under the fuzzer it was a crash with no property. With an analysis mode set, both now return and record nothing, in the libFuzzer and KLEE generators (shared record_verifier_error). With the mode unset (direct API users) they keep recording REACHABILITY and aborting. Tests: libFuzzer unit tests (memtrack: __VERIFIER_error then an invalid load gives MemsafetyDeref; target: only map2check_reach_target reports) and tests/klee programs vererr_deref.c (FALSE-DEREF on both runtimes) and vererr_target.c (no abort, NONE, 2 paths on both). SD-08 rewritten: its legacy paragraph described map2check_error, not __VERIFIER_error. --- docs/semantic-deltas.md | 48 ++++++++++----- rust/src/nondet/klee.rs | 15 +++-- rust/src/nondet/libfuzzer.rs | 109 ++++++++++++++++++++++++++++++++--- rust/src/nondet/mod.rs | 24 ++++++-- tests/klee/m2c.h | 7 +++ tests/klee/vererr_deref.c | 30 ++++++++++ tests/klee/vererr_target.c | 18 ++++++ tests/klee_paths.sh | 34 ++++++++++- 8 files changed, 251 insertions(+), 34 deletions(-) create mode 100644 tests/klee/vererr_deref.c create mode 100644 tests/klee/vererr_target.c diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 657a5f2..a73e440 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -273,24 +273,46 @@ int __map2check_main__(void) { /* LLVMFuzzerTestOneInput returns 0 for every input; the process goes on */ ``` -### SD-08 — `__VERIFIER_error()` reports REACHABILITY, then aborts - -**Classification:** intentional -**Tests:** `rust/src/nondet/libfuzzer.rs::verifier_error_aborts_and_reports_reachability`, `rust/src/nondet/libfuzzer.rs::verifier_assert_false_aborts_and_reports_reachability` - -**Rust:** `__VERIFIER_error()` and `__VERIFIER_assert(0)` record -`REACHABILITY`, print the JSON report (only when no analysis mode is -set, SD-01) and call `abort()`. LibFuzzer treats -the abort as a crash. - -**Legacy (embedded C library):** a violation ends in `abort()` through -`map2check_error()` (`lib_old/Map2CheckFunctions.c:72-102`). +### SD-08 — `__VERIFIER_error()` / `__VERIFIER_assert(0)` without an analysis mode + +**Classification:** fixed (#20) +**Tests:** `rust/src/nondet/libfuzzer.rs::verifier_error_is_a_no_op_with_a_mode_and_the_deref_is_reported`, `rust/src/nondet/libfuzzer.rs::target_mode_reaches_only_through_map2check_reach_target`, `rust/src/nondet/libfuzzer.rs::verifier_error_aborts_and_reports_reachability`, `rust/src/nondet/libfuzzer.rs::verifier_assert_false_aborts_and_reports_reachability` + +**Legacy (develop):** both are empty. `__VERIFIER_error()` is +`// DO NOTHING` (`Map2CheckFunctions.c:57-59`) and `__VERIFIER_assert` is +`// DO NOTHING` (`AnalysisModeAssert.c:25-27`, defined only in the assert +mode). The passes carry the checks: the target pass emits +`map2check_target_function` (reachability), the assert pass emits +`map2check_assert`. The earlier text of this entry, "a violation ends in +`abort()` through `map2check_error()`", described `map2check_error`, not +`__VERIFIER_error`. + +**Rust, mode set (a Map2Check run):** the same as develop: both return and +record nothing, in the libFuzzer and the KLEE generators. Reachability is +reported only by `map2check_reach_target`, an assert failure only by +`map2check_is_valid_assert`. Before this fix both recorded `REACHABILITY` +and aborted with no property file. Under KLEE's +`--exit-on-error-type=Abort` that stopped the exploration before the real +violation of a memory or overflow task. + +Under KLEE the same is checked by `tests/klee_paths.sh` with +`tests/klee/vererr_deref.c` and `tests/klee/vererr_target.c`. Fixed for the +mode-set path in the Task 2.7 review; the mode-unset path below stays as +an intentional delta. + +**Rust, mode unset (direct API users):** `__VERIFIER_error()` and +`__VERIFIER_assert(0)` record `REACHABILITY`, print the JSON report +(SD-01) and call `abort()`; LibFuzzer treats the abort as a crash. **Reproduce (link with `--features libfuzzer`):** ```c map2check_init(); -__VERIFIER_error(); /* prints "REACHABILITY", then SIGABRT */ +__VERIFIER_error(); /* no mode: prints "REACHABILITY", then SIGABRT */ + +map2check_set_mode(1); /* memtrack */ +map2check_init(); +__VERIFIER_error(); /* returns; nothing recorded */ ``` ### SD-09 — Invalid arguments are ignored instead of causing undefined behaviour diff --git a/rust/src/nondet/klee.rs b/rust/src/nondet/klee.rs index cb20557..4262ce0 100644 --- a/rust/src/nondet/klee.rs +++ b/rust/src/nondet/klee.rs @@ -191,16 +191,19 @@ pub extern "C" fn __VERIFIER_assume(condition: i32) { ); } -/// SV-COMP reachability primitive, as in `libfuzzer.rs`: records a -/// `Reachability` violation and aborts, which KLEE reports as an abort -/// error. +/// SV-COMP reachability primitive, as in `libfuzzer.rs`: with an analysis +/// mode set it does nothing, as develop's (`Map2CheckFunctions.c:57-59`), +/// so KLEE keeps exploring; with none it records `Reachability` and aborts +/// (SD-08). #[no_mangle] pub extern "C" fn __VERIFIER_error() { - crate::ffi::ffi_guard(super::record_verifier_error, ()); - std::process::abort(); + if crate::ffi::ffi_guard(super::record_verifier_error, false) { + std::process::abort(); + } } -/// SV-COMP assert primitive: `__VERIFIER_error()` when `condition` is false. +/// SV-COMP assert primitive: `__VERIFIER_error()` when `condition` is false +/// (a no-op with a mode set, as develop's `AnalysisModeAssert.c:25-27`). #[no_mangle] pub extern "C" fn __VERIFIER_assert(condition: i32) { crate::ffi::ffi_guard( diff --git a/rust/src/nondet/libfuzzer.rs b/rust/src/nondet/libfuzzer.rs index 2d9127d..f4442d5 100644 --- a/rust/src/nondet/libfuzzer.rs +++ b/rust/src/nondet/libfuzzer.rs @@ -314,17 +314,21 @@ pub extern "C-unwind" fn map2check_fuzzer_assume(condition: i32) { } } -/// SV-COMP reachability primitive: records a `Reachability` violation and -/// aborts the process. Reaching this function at runtime means the property -/// being checked was violated; it never returns. +/// SV-COMP reachability primitive. With an analysis mode set it does +/// nothing, as in develop: the target pass reports reachability through +/// `map2check_reach_target`. With the mode unset it records a +/// `Reachability` violation, prints the JSON report and aborts (SD-08). #[no_mangle] pub extern "C" fn __VERIFIER_error() { - crate::ffi::ffi_guard(super::record_verifier_error, ()); - std::process::abort(); + if crate::ffi::ffi_guard(super::record_verifier_error, false) { + std::process::abort(); + } } -/// SV-COMP assert primitive: calls `__VERIFIER_error()` (and thus aborts) -/// when `condition` is false; otherwise a no-op. +/// SV-COMP assert primitive: calls `__VERIFIER_error()` when `condition` +/// is false, which with an analysis mode set does nothing (develop's +/// `__VERIFIER_assert` is empty; the assert pass emits +/// `map2check_is_valid_assert`) and with none aborts (SD-08). #[no_mangle] pub extern "C" fn __VERIFIER_assert(condition: i32) { crate::ffi::ffi_guard( @@ -894,4 +898,95 @@ mod tests { fn verifier_assert_false_aborts_and_reports_reachability() { assert_aborted_with_reachability(&run_child("verifier_assert_false_child")); } + + // With an analysis mode set (a Map2Check run), `__VERIFIER_error` and + // `__VERIFIER_assert` do nothing, as in develop (`Map2CheckFunctions.c:57-59`, + // `AnalysisModeAssert.c:25-27`): the passes carry the target and assert + // checks (`map2check_reach_target`, `map2check_is_valid_assert`). + + /// Runs a child that must survive, and returns its stdout. + fn run_surviving_child(name: &str) -> String { + let exe = std::env::current_exe().unwrap(); + let output = std::process::Command::new(exe) + .args(["--exact", &format!("nondet::libfuzzer::tests::{name}")]) + .args(["--nocapture", "--test-threads=1"]) + .env(CHILD_ENV, "1") + .output() + .unwrap(); + let stdout = String::from_utf8_lossy(&output.stdout).into_owned(); + assert!( + output.status.success(), + "`{name}` não terminou bem:\n{stdout}" + ); + assert!( + stdout.contains(STARTED), + "o filho não rodou `{name}`:\n{stdout}" + ); + assert!( + stdout.contains(SURVIVED), + "`{name}` encerrou o processo:\n{stdout}" + ); + stdout + } + + fn current_property() -> crate::caller::ViolatedProperty { + crate::ffi::get_current_property() + } + + #[test] + fn verifier_error_with_memtrack_mode_child() { + if is_child() { + println!("{STARTED}"); + crate::ffi::map2check_set_mode(1); // memtrack + crate::ffi::map2check_init(); + __VERIFIER_error(); + __VERIFIER_assert(0); + println!("after-error={:?}", current_property()); + let block = [0_i32; 4]; + let base = block.as_ptr().cast::(); + // SAFETY: only the address is recorded and checked; nothing is read. + unsafe { + crate::ffi::map2check_map_malloc(base, 16); + crate::ffi::map2check_check_load( + block.as_ptr().wrapping_add(4).cast(), + 17, + 0, + 4, + b"main\0".as_ptr().cast(), + ); + } + println!("after-deref={:?}", current_property()); + println!("{SURVIVED}"); + } + } + + #[test] + fn verifier_error_is_a_no_op_with_a_mode_and_the_deref_is_reported() { + let out = run_surviving_child("verifier_error_with_memtrack_mode_child"); + assert!(out.contains("after-error=None"), "{out}"); + assert!(out.contains("after-deref=MemsafetyDeref"), "{out}"); + } + + #[test] + fn verifier_error_with_target_mode_child() { + if is_child() { + println!("{STARTED}"); + crate::ffi::map2check_set_mode(5); // target + crate::ffi::map2check_init(); + __VERIFIER_error(); + __VERIFIER_assert(0); + println!("after-error={:?}", current_property()); + // SAFETY: NUL-terminated literal. + unsafe { crate::ffi::map2check_reach_target(b"reach_error\0".as_ptr().cast(), 0, 12) }; + println!("after-target={:?}", current_property()); + println!("{SURVIVED}"); + } + } + + #[test] + fn target_mode_reaches_only_through_map2check_reach_target() { + let out = run_surviving_child("verifier_error_with_target_mode_child"); + assert!(out.contains("after-error=None"), "{out}"); + assert!(out.contains("after-target=Reachability"), "{out}"); + } } diff --git a/rust/src/nondet/mod.rs b/rust/src/nondet/mod.rs index c4743cf..d7c43bf 100644 --- a/rust/src/nondet/mod.rs +++ b/rust/src/nondet/mod.rs @@ -109,11 +109,24 @@ pub mod libfuzzer; #[cfg(feature = "klee")] pub mod klee; -/// Body of `__VERIFIER_error`, the same for every generator: records a -/// `Reachability` violation and, without a mode, prints the JSON report. -/// The export aborts afterwards. +/// Body of `__VERIFIER_error`, the same for every generator. Returns +/// whether the export must abort. +/// +/// With an analysis mode set (a Map2Check run) it does nothing, as develop's +/// `__VERIFIER_error` (`Map2CheckFunctions.c:57-59`) and `__VERIFIER_assert` +/// (`AnalysisModeAssert.c:25-27`): there the passes carry the checks, the +/// target through `map2check_reach_target` and asserts through +/// `map2check_is_valid_assert`. An abort here would end the run with no +/// property recorded, and under KLEE's `--exit-on-error-type=Abort` stop the +/// exploration before the real violation (SD-08). +/// +/// With the mode unset (direct API users), records a `Reachability` +/// violation, prints the JSON report and asks the export to abort. #[cfg(any(feature = "libfuzzer", feature = "klee"))] -fn record_verifier_error() { +fn record_verifier_error() -> bool { + if crate::state::config().mode.is_some() { + return false; + } if let Err(e) = crate::state::with_state(|s| { s.set_false( crate::caller::ViolatedProperty::Reachability, @@ -124,8 +137,9 @@ fn record_verifier_error() { }) { eprintln!("[map2check] error: {e}"); } - // Out of the lock, and only without a mode (#19 ruling). + // Out of the lock (#19 ruling). crate::output::print_report_unless_mode_set(); + true } #[cfg(test)] diff --git a/tests/klee/m2c.h b/tests/klee/m2c.h index b80e880..05b7620 100644 --- a/tests/klee/m2c.h +++ b/tests/klee/m2c.h @@ -12,6 +12,13 @@ #include +/* SV-COMP primitives. With a mode set both runtimes treat them as no-ops + * (develop: Map2CheckFunctions.c:57-59, AnalysisModeAssert.c:25-27); the + * passes report through map2check_*. develop defines __VERIFIER_assert only + * in assert mode, so the programs call it only on the Rust runtime. */ +void __VERIFIER_error(void); +void __VERIFIER_assert(int condition); + #define M2C_MODE_MEMTRACK 1 #define M2C_MODE_TARGET 5 diff --git a/tests/klee/vererr_deref.c b/tests/klee/vererr_deref.c new file mode 100644 index 0000000..9a7ac29 --- /dev/null +++ b/tests/klee/vererr_deref.c @@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Memtrack mode: __VERIFIER_error() (and, on the Rust runtime, a false + * __VERIFIER_assert) is reached before the off-by-one read. With a mode set + * both are no-ops, as in develop, so KLEE (stopping at the first abort, as + * the frontend runs it) must still find the read: FALSE-DEREF, not an + * abort with no property. */ +#include +#include "m2c.h" + +int __map2check_main__(void) { + M2C_START(M2C_MODE_MEMTRACK); + int *p = malloc(4 * sizeof(int)); + TRACK_MALLOC(p, (int)(4 * sizeof(int)), 13); + for (int k = 0; k < 4; k++) + p[k] = k; + int i = NONDET(int, int, 16); + __VERIFIER_error(); +#ifndef M2C_DEVELOP + __VERIFIER_assert(0); +#endif + int v = 0; + if (i >= 0 && i <= 4) { + CHECK_LOAD(&p[i], (int)sizeof(int), 23); + v = p[i]; + } + TRACK_FREE(p, 26); + free(p); + map2check_success(); + return v; +} diff --git a/tests/klee/vererr_target.c b/tests/klee/vererr_target.c new file mode 100644 index 0000000..d8262e5 --- /dev/null +++ b/tests/klee/vererr_target.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Target mode: __VERIFIER_error() alone reports nothing. Reachability comes + * only from map2check_reach_target, which the target pass emits (reach.c). + * Expected: two paths, no abort, map2check_property NONE. */ +#include "m2c.h" + +int __map2check_main__(void) { + M2C_START(M2C_MODE_TARGET); + int x = NONDET(int, int, 8); + if (x == 42) { + __VERIFIER_error(); +#ifndef M2C_DEVELOP + __VERIFIER_assert(0); +#endif + } + map2check_success(); + return 0; +} diff --git a/tests/klee_paths.sh b/tests/klee_paths.sh index ddc71dd..5a01683 100755 --- a/tests/klee_paths.sh +++ b/tests/klee_paths.sh @@ -19,6 +19,10 @@ # (the spike's 2 -> 22 case, condition 7); # deref.c KLEE finds the off-by-one read and the property file says # FALSE-DEREF; +# vererr_deref.c, vererr_target.c __VERIFIER_error()/_assert(0) do +# nothing with a mode set, as in develop: memtrack still ends in +# FALSE-DEREF, target mode reports nothing without +# map2check_reach_target; # layout.c condition 9: the ktest of the error path, turned into a seed # as the frontend's exportKleeVectorsAsSeeds does, replayed # through the Rust libFuzzer reader (--features libfuzzer, @@ -111,7 +115,7 @@ develop_runtime() { # run_klee() { local prog="$1" rt="$2" kind="$3" dir="$W/$1-$2-$3" rtbc cflags=() mode=target mkdir -p "$dir" - [ "$prog" = deref ] && mode=memtrack + case "$prog" in deref | vererr_deref) mode=memtrack ;; esac if [ "$rt" = develop ]; then cflags=(-DM2C_DEVELOP) rtbc="$W/dev-$mode.bc" @@ -248,6 +252,30 @@ else fail "deref.c: the abort's ktest does not have i = 4" fi +# ── vererr_*.c: __VERIFIER_error/_assert are no-ops with a mode set ───────── +echo "== vererr_deref.c" +run_klee vererr_deref rust full +count vererr_deref rust +run_klee vererr_deref rust frontend +d="$W/vererr_deref-rust-frontend" +if [ "$(head -n 1 "$d/map2check_property" 2>/dev/null)" = FALSE-DEREF ] && + k="$(abort_ktest "$d")" && [ -n "$k" ] && + ktest_objects "$k" "$W/seed.tmp" | grep -qx 'non_det_int 4 04000000'; then + pass "vererr_deref.c: __VERIFIER_error is a no-op in memtrack mode; FALSE-DEREF with i = 4" +else + fail "vererr_deref.c: property '$(cat "$d/map2check_property" 2>/dev/null)'" +fi +echo "== vererr_target.c" +run_klee vererr_target rust full +count vererr_target rust +d="$W/vererr_target-rust-full" +if [ "$PATHS_COMPLETED" = 2 ] && [ "$PATHS_PARTIAL" = 0 ] && [ -z "$(abort_ktest "$d")" ] && + [ "$(head -n 1 "$d/map2check_property" 2>/dev/null)" = NONE ]; then + pass "vererr_target.c: __VERIFIER_error alone reports nothing (2 paths, no abort, NONE)" +else + fail "vererr_target.c: $PATHS_COMPLETED+$PATHS_PARTIAL paths, property '$(cat "$d/map2check_property" 2>/dev/null)'" +fi + # ── layout.c: ktest -> seed -> Rust libFuzzer reader (condition 9) ──────────── echo "== layout.c" run_klee layout rust full @@ -303,7 +331,7 @@ fi # ── develop's C runtime, same programs ──────────────────────────────────────── if [ -n "${DEVELOP_SRC:-}" ]; then echo "== develop C runtime ($DEVELOP_SRC)" - for prog in prog reach range deref layout; do + for prog in prog reach range deref vererr_deref vererr_target layout; do run_klee "$prog" develop full count "$prog" develop run_klee "$prog" develop frontend @@ -315,7 +343,7 @@ fi table="$( echo "| program | Rust runtime (completed+partial) | develop C runtime (completed+partial) |" echo "|---|---|---|" - for prog in prog reach range json deref layout; do + for prog in prog reach range json deref vererr_deref vererr_target layout; do echo "| $prog.c | ${COUNT[$prog/rust]:-?} | ${COUNT[$prog/develop]:-not run} |" done )" From 12dbb6fcae2ab55b7bc6abc9a13134d99d7bf355 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:33:15 -0400 Subject: [PATCH 44/63] build: gate the KLEE runtime on intrinsics KLEE 3.1 executes - klee-lower-intrinsics.py: after the rewrite, every intrinsic still declared must be in KLEE 3.1's list (the `case Intrinsic::` labels of Executor.cpp and IntrinsicCleaner.cpp at v3.1), with bitreverse as a justified exception; anything else fails the build instead of killing a KLEE path at run time. The hand-written rewrite stays the only mechanism: -expand-reductions covers only the reduce family and lives in the legacy pass manager. - build-klee-runtime.sh runs the lowering's self-tests before using it: lli-16 equivalence on tests/klee/lower-intrinsics.ll (now with reduce.add.v2i64, which the runtime uses) and rejection of tests/klee/unknown-intrinsic.ll (llvm.masked.load). It builds in a temporary directory, removes first and moves the result only at the end, so a failure leaves no bitcode; klee_paths.sh checks that. - CI klee_runtime: clippy 1.72.1 on the klee build (the only lint of nondet/klee.rs) and lli-16 for the self-tests. - klee.rs: main allows not_unsafe_ptr_arg_deref (argv is only passed on). --- .github/workflows/CI.yml | 15 ++++++-- rust/src/nondet/klee.rs | 3 ++ scripts/build-klee-runtime.sh | 57 ++++++++++++++++++++--------- scripts/klee-lower-intrinsics.py | 63 +++++++++++++++++++++++++++----- tests/klee/lower-intrinsics.ll | 10 +++-- tests/klee/unknown-intrinsic.ll | 8 ++++ tests/klee_paths.sh | 17 ++++++--- 7 files changed, 134 insertions(+), 39 deletions(-) create mode 100644 tests/klee/unknown-intrinsic.ll diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index 263937f..2ffb6bf 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -171,12 +171,21 @@ jobs: - name: Baixar o código do repositório uses: actions/checkout@v4 - - name: Instalar as ferramentas do LLVM 16 (opt, llvm-dis, llvm-nm, llvm-as) + - name: Instalar as ferramentas do LLVM 16 (opt, llvm-dis, llvm-nm, llvm-as, lli) run: sudo apt-get update && sudo apt-get install -y llvm-16 - - name: Instalar o rustc 1.72.1 (LLVM 16) - run: rustup toolchain install 1.72.1 --profile minimal + - name: Instalar o rustc 1.72.1 (LLVM 16) com clippy + run: rustup toolchain install 1.72.1 --profile minimal --component clippy + # O clippy em 1.82 do job acima não compila nondet/klee.rs (feature + # `klee` só existe em klee/Cargo.toml); este é o lint dele. + - name: Clippy do build KLEE (rustc 1.72.1, feature klee) + run: RUSTUP_TOOLCHAIN=1.72.1 cargo clippy --locked --manifest-path klee/Cargo.toml --features klee -- -D warnings + + # O script roda os autotestes da troca de intrínsecos antes de usá-la: + # equivalência por lli-16 em tests/klee/lower-intrinsics.ll e recusa + # de tests/klee/unknown-intrinsic.ll; qualquer intrínseco que o KLEE + # 3.1 não execute faz o build falhar. - name: Compilar o runtime KLEE em bitcode (klee/Cargo.toml, --locked) run: scripts/build-klee-runtime.sh target/klee/map2check-klee.bc diff --git a/rust/src/nondet/klee.rs b/rust/src/nondet/klee.rs index 4262ce0..e49a79a 100644 --- a/rust/src/nondet/klee.rs +++ b/rust/src/nondet/klee.rs @@ -219,6 +219,9 @@ pub extern "C" fn __VERIFIER_assert(condition: i32) { /// KLEE's entry point. develop's KLEE generator defines `main` the same way /// (`NonDetGeneratorKlee.c:16`): the passes renamed the program's `main` to /// `__map2check_main__` and inserted the mode, init and success calls. +// `argv` is only handed on to the program's own `main`, never dereferenced +// here; the signature is the C one, so the function cannot be `unsafe`. +#[allow(clippy::not_unsafe_ptr_arg_deref)] #[no_mangle] pub extern "C" fn main(argc: c_int, argv: *mut *mut c_char) -> c_int { crate::ffi::ffi_guard( diff --git a/scripts/build-klee-runtime.sh b/scripts/build-klee-runtime.sh index 6b956a7..278b22d 100755 --- a/scripts/build-klee-runtime.sh +++ b/scripts/build-klee-runtime.sh @@ -14,11 +14,12 @@ # # Environment: # KLEE_RUSTC_TOOLCHAIN rustup toolchain (default 1.72.1) -# OPT, LLVM_DIS, LLVM_NM, LLVM_AS LLVM 16 tools (default opt-16, -# llvm-dis-16, llvm-nm-16, llvm-as-16) +# OPT, LLVM_DIS, LLVM_NM, LLVM_AS, LLI LLVM 16 tools (default opt-16, +# llvm-dis-16, llvm-nm-16, llvm-as-16, lli-16) # CARGO_TARGET_DIR target directory (default /target/klee) # -# Every step must succeed; the script stops at the first failure. +# Every step must succeed; the script stops at the first failure and then +# leaves nothing at . set -euo pipefail die() { @@ -33,15 +34,19 @@ case "$out" in *) die "the output must end in .bc: $out" ;; esac +# A failed build must not leave an earlier (or partial) runtime behind. +rm -f "$out" + ROOT="$(cd "$(dirname "$0")/.." && pwd)" TOOLCHAIN="${KLEE_RUSTC_TOOLCHAIN:-1.72.1}" OPT="${OPT:-opt-16}" LLVM_DIS="${LLVM_DIS:-llvm-dis-16}" LLVM_NM="${LLVM_NM:-llvm-nm-16}" LLVM_AS="${LLVM_AS:-llvm-as-16}" +LLI="${LLI:-lli-16}" TARGET_DIR="${CARGO_TARGET_DIR:-$ROOT/target/klee}" -for tool in cargo rustc python3 "$OPT" "$LLVM_DIS" "$LLVM_NM" "$LLVM_AS"; do +for tool in cargo rustc python3 "$OPT" "$LLVM_DIS" "$LLVM_NM" "$LLVM_AS" "$LLI"; do command -v "$tool" >/dev/null || die "$tool not found in PATH" done @@ -64,8 +69,9 @@ if [ -d "$deps" ]; then find "$deps" -maxdepth 1 -name 'map2check-*.bc' -delete fi -log="$(mktemp)" -trap 'rm -f "$log" "$log.ll" "$log.lowered.ll"' EXIT +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT +log="$work/cargo.log" # --locked: the dependency graph is the one in klee/Cargo.lock (v3), never # a fresh resolution. The profile (klee/Cargo.toml) sets panic = abort, # lto = fat and codegen-units = 1; the check below proves rustc got them. @@ -94,30 +100,45 @@ mapfile -t modules < <(find "$deps" -maxdepth 1 -name 'map2check-*.bc') # The debug info of the precompiled std does not survive llvm-link-16 + # KLEE ("Broken module found"); only the runtime loses it, the analysed # program keeps its -g. -mkdir -p "$(dirname "$out")" -"$OPT" --strip-debug "${modules[0]}" -o "$out" || die "$OPT --strip-debug failed" -"$LLVM_DIS" "$out" -o /dev/null || die "$LLVM_DIS cannot read $out" +# Everything is built in $work and moved to $out only at the end, so a +# failed step never leaves a bitcode behind. +"$OPT" --strip-debug "${modules[0]}" -o "$work/stripped.bc" || die "$OPT --strip-debug failed" +"$LLVM_DIS" "$work/stripped.bc" -o "$work/stripped.ll" || die "$LLVM_DIS cannot read the stripped module" # KLEE 3.1 cannot execute the vector/x86 intrinsics of the precompiled std # ("unimplemented intrinsic" in core::fmt padding): replace them by scalar -# functions. Fails on an intrinsic the script does not know. -"$LLVM_DIS" "$out" -o "$log.ll" || die "$LLVM_DIS cannot read $out" -python3 -I "$ROOT/scripts/klee-lower-intrinsics.py" "$log.ll" "$log.lowered.ll" || - die "lowering the vector intrinsics failed" -"$LLVM_AS" "$log.lowered.ll" -o "$out" || die "$LLVM_AS rejected the lowered module" -rm -f "$log.ll" "$log.lowered.ll" -"$LLVM_DIS" "$out" -o /dev/null || die "$LLVM_DIS cannot read $out" +# functions. The script fails on any remaining intrinsic KLEE 3.1 does not +# execute. Its two self-tests run first: the scalar versions must compute +# what the intrinsics compute (lli-16), and an unknown intrinsic must be +# rejected. +lower="$ROOT/scripts/klee-lower-intrinsics.py" +fixtures="$ROOT/tests/klee" +python3 -I "$lower" "$fixtures/lower-intrinsics.ll" "$work/fixture.ll" >/dev/null || + die "the lowering self-test fixture was rejected" +expected="$("$LLI" "$fixtures/lower-intrinsics.ll")" || die "$LLI failed on the fixture" +got="$("$LLI" "$work/fixture.ll")" || die "$LLI failed on the lowered fixture" +[ "$expected" = "$got" ] || + die "the scalar versions differ from the intrinsics: '$expected' vs '$got'" +if python3 -I "$lower" "$fixtures/unknown-intrinsic.ll" "$work/unknown.ll" >/dev/null 2>&1; then + die "the lowering accepted an intrinsic KLEE 3.1 cannot execute (unknown-intrinsic.ll)" +fi +python3 -I "$lower" "$work/stripped.ll" "$work/lowered.ll" || + die "lowering the intrinsics failed" +"$LLVM_AS" "$work/lowered.ll" -o "$work/runtime.bc" || die "$LLVM_AS rejected the lowered module" +"$LLVM_DIS" "$work/runtime.bc" -o /dev/null || die "$LLVM_DIS cannot read the lowered module" # The module must define what the symex pipeline links against, and leave # the KLEE intrinsics to KLEE. -defined="$("$LLVM_NM" --defined-only "$out")" +defined="$("$LLVM_NM" --defined-only "$work/runtime.bc")" for sym in main __VERIFIER_nondet_int __VERIFIER_nondet_double __VERIFIER_assume \ __VERIFIER_error map2check_init map2check_set_mode map2check_success; do grep -Eq "[[:space:]]$sym\$" <<<"$defined" || die "$out does not define $sym" done -undefined="$("$LLVM_NM" --undefined-only "$out")" +undefined="$("$LLVM_NM" --undefined-only "$work/runtime.bc")" for sym in klee_make_symbolic klee_assume __map2check_main__; do grep -Eq "[[:space:]]$sym\$" <<<"$undefined" || die "$out does not import $sym" done +mkdir -p "$(dirname "$out")" +mv -f "$work/runtime.bc" "$out" || die "cannot write $out" echo "build-klee-runtime: wrote $out (rustc $TOOLCHAIN, LLVM $llvm_version)" diff --git a/scripts/klee-lower-intrinsics.py b/scripts/klee-lower-intrinsics.py index 26061f9..c662ff0 100644 --- a/scripts/klee-lower-intrinsics.py +++ b/scripts/klee-lower-intrinsics.py @@ -12,12 +12,16 @@ internal function with the same signature and semantics, written with extractelement/insertelement and scalar arithmetic, which KLEE supports. -Any other `llvm.vector.*` or `llvm.x86.*` declaration is an error: a new -rustc brought one this script does not know, and it must be added here. +After the rewrite, every intrinsic still declared must be one KLEE 3.1 +executes (KLEE_31_INTRINSICS below); any other one is an error, because +under KLEE it would kill a path silently ("unimplemented intrinsic") and a +killed path can hide a violation. A new rustc or std that brings one must +add a scalar version here, or prove KLEE runs it. -`llvm.bitreverse.*` (miniz_oxide, which only std's backtrace printing -reaches) is left alone: KLEE warns "unsupported intrinsic" at load, and its -own --optimize would fold a scalar version back into the intrinsic anyway. +Why a rewrite and not `opt-16 -enable-new-pm=0 -expand-reductions`: that +pass only covers the reduce family (pmulhu.w and pause would still need +this script) and lives in the legacy pass manager LLVM is removing, so one +mechanism, checked by `lli-16` on tests/klee/lower-intrinsics.ll, is kept. Usage: klee-lower-intrinsics.py """ import re @@ -31,7 +35,45 @@ ) PMULHU = re.compile(r"^declare\s+<8 x i16>\s+@(?Pllvm\.x86\.sse2\.pmulhu\.w)\([^\n]*$", re.M) PAUSE = re.compile(r"^declare\s+void\s+@(?Pllvm\.x86\.sse2\.pause)\(\)[^\n]*$", re.M) -LEFTOVER = re.compile(r"^declare[^\n]*@(llvm\.(?:vector|x86)\.[\w.]+)", re.M) +DECLARED = re.compile(r"^declare[^\n]*?@(llvm\.[\w.]+)\(", re.M) + +# Intrinsics KLEE 3.1 executes: the `case Intrinsic::` labels of +# lib/Core/Executor.cpp (executeCall) and lib/Module/IntrinsicCleaner.cpp +# (lowered before execution) at tag v3.1, `_` written as `.`. A declared +# intrinsic matches when its name is one of these or one of these followed +# by `.`. +KLEE_31_INTRINSICS = """ +abs addressofreturnaddress annotation assume bswap ceil copysign cos ctlz +ctpop cttz dbg.declare dbg.label dbg.value eh.typeid.for exp exp2 expect +experimental.noalias.scope.decl fabs floor flt.rounds fma fmuladd +frameaddress fshl fshr get.dynamic.area.offset get.rounding invariant.end +invariant.start is.constant lifetime.end lifetime.start log log10 log2 +memcpy memmove memset objectsize pcmarker pow prefetch ptr.annotation +readcyclecounter returnaddress round roundeven sadd.sat sadd.with.overflow +sin smax smin smul.with.overflow sqrt ssub.sat ssub.with.overflow +stackrestore stacksave trap trunc uadd.sat uadd.with.overflow umax umin +umul.with.overflow usub.sat usub.with.overflow vacopy vaend var.annotation +vastart +""".split() + +# Declared but accepted although KLEE 3.1 cannot run them, each with why it +# is never executed by the runtime. +ACCEPTED = { + # miniz_oxide, reached only by std's backtrace printing, which + # panic = "abort" never runs. KLEE warns "unsupported intrinsic" at + # load; KLEE's --optimize folds any scalar version back into it. + "bitreverse": "backtrace symbolisation only", +} + + +def unsupported(names): + allowed = KLEE_31_INTRINSICS + list(ACCEPTED) + bad = [] + for name in names: + base = name[len("llvm."):] + if not any(base == a or base.startswith(a + ".") for a in allowed): + bad.append(name) + return sorted(set(bad)) def new_name(name): @@ -104,9 +146,12 @@ def main(): for name in renamed: ir = re.sub(r"@" + re.escape(name) + r"(?![\w.])", "@" + new_name(name), ir) - leftover = sorted(set(LEFTOVER.findall(ir))) - if leftover: - sys.exit("klee-lower-intrinsics: no scalar version of: " + ", ".join(leftover)) + bad = unsupported(DECLARED.findall(ir)) + if bad: + sys.exit( + "klee-lower-intrinsics: KLEE 3.1 cannot execute, and there is no " + "scalar version of: " + ", ".join(bad) + ) with open(sys.argv[2], "w") as f: f.write(ir) diff --git a/tests/klee/lower-intrinsics.ll b/tests/klee/lower-intrinsics.ll index 3a000ef..22c0299 100644 --- a/tests/klee/lower-intrinsics.ll +++ b/tests/klee/lower-intrinsics.ll @@ -1,15 +1,19 @@ ; SPDX-License-Identifier: GPL-3.0-only -; tests/klee_paths.sh runs this with lli-16 before and after -; scripts/klee-lower-intrinsics.py: the outputs must be identical. +; scripts/build-klee-runtime.sh (and so the CI job klee_runtime) runs this +; with lli-16 before and after scripts/klee-lower-intrinsics.py: the outputs +; must be identical, or the build fails. declare i64 @llvm.vector.reduce.add.v4i64(<4 x i64>) declare i64 @llvm.vector.reduce.or.v2i64(<2 x i64>) +declare i64 @llvm.vector.reduce.add.v2i64(<2 x i64>) declare <8 x i16> @llvm.x86.sse2.pmulhu.w(<8 x i16>, <8 x i16>) declare void @llvm.x86.sse2.pause() declare i32 @printf(ptr, ...) @f = private constant [29 x i8] c"%lld %lld %d %d %d %d %d %d\0A\00" define i32 @main() { %a = call i64 @llvm.vector.reduce.add.v4i64(<4 x i64> ) - %b = call i64 @llvm.vector.reduce.or.v2i64(<2 x i64> ) + %b0 = call i64 @llvm.vector.reduce.or.v2i64(<2 x i64> ) + %c = call i64 @llvm.vector.reduce.add.v2i64(<2 x i64> ) + %b = add i64 %b0, %c %m = call <8 x i16> @llvm.x86.sse2.pmulhu.w(<8 x i16> , <8 x i16> ) %m0 = extractelement <8 x i16> %m, i32 0 %m1 = extractelement <8 x i16> %m, i32 1 diff --git a/tests/klee/unknown-intrinsic.ll b/tests/klee/unknown-intrinsic.ll new file mode 100644 index 0000000..785dd44 --- /dev/null +++ b/tests/klee/unknown-intrinsic.ll @@ -0,0 +1,8 @@ +; SPDX-License-Identifier: GPL-3.0-only +; scripts/klee-lower-intrinsics.py must reject this module: KLEE 3.1 does +; not implement llvm.masked.load, and there is no scalar version of it. +declare <2 x i64> @llvm.masked.load.v2i64.p0(ptr, i32, <2 x i1>, <2 x i64>) +define <2 x i64> @f(ptr %p) { + %v = call <2 x i64> @llvm.masked.load.v2i64.p0(ptr %p, i32 8, <2 x i1> , <2 x i64> zeroinitializer) + ret <2 x i64> %v +} diff --git a/tests/klee_paths.sh b/tests/klee_paths.sh index 5a01683..4e7d5e5 100755 --- a/tests/klee_paths.sh +++ b/tests/klee_paths.sh @@ -72,13 +72,18 @@ fail() { KLEE_FLAGS=(--external-calls=all --optimize --use-cex-cache --solver-backend=z3 --libc=uclibc --max-time=300s) -echo "== scripts/klee-lower-intrinsics.py keeps the intrinsics' semantics" -python3 -I "$ROOT/scripts/klee-lower-intrinsics.py" "$T/lower-intrinsics.ll" "$W/lowered.ll" >/dev/null -if [ "$(lli-16 "$T/lower-intrinsics.ll")" = "$(lli-16 "$W/lowered.ll")" ] && - ! grep -q '@llvm\.\(vector\|x86\)' "$W/lowered.ll"; then - pass "lower-intrinsics.ll: same output from the intrinsics and their scalar versions" +# The lowering's self-tests (lli-16 equivalence on tests/klee/ +# lower-intrinsics.ll, rejection of tests/klee/unknown-intrinsic.ll) run +# inside scripts/build-klee-runtime.sh, so the CI build runs them too. +echo "== a failing build leaves no bitcode" +touch "$W/broken.bc" +if LLI=false CARGO_TARGET_DIR="$ROOT/target/klee-docker" \ + "$ROOT/scripts/build-klee-runtime.sh" "$W/broken.bc" >"$W/broken.log" 2>&1; then + fail "build-klee-runtime.sh succeeded with a broken lli" +elif [ -e "$W/broken.bc" ]; then + fail "build-klee-runtime.sh failed but left $W/broken.bc" else - fail "lower-intrinsics.ll: the scalar versions differ from the intrinsics" + pass "build-klee-runtime.sh fails on a broken lowering self-test and leaves no .bc" fi echo "== building the KLEE runtime" From d39f5818f7eb999699a6a0915f36ccd1822b8abd Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:33:15 -0400 Subject: [PATCH 45/63] docs: document the KLEE exports and the review fixes api-deltas: __VERIFIER_error/_assert are libfuzzer and klee exports and do nothing with a mode set (SD-08); new section on the KLEE generator and its differences from develop's (naming, float, pchar cap, concretisation). klee-feasibility: __VERIFIER_error row, the vererr programs and counts, the intrinsic allowlist and self-tests, the clippy step. --- docs/api-deltas.md | 33 +++++++++++++++++++++++++++++---- docs/klee-feasibility.md | 10 ++++++---- 2 files changed, 35 insertions(+), 8 deletions(-) diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 6116edd..9e3f0d4 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -126,8 +126,8 @@ build with `--features libfuzzer`. | `__VERIFIER_nondet_pchar` | `char *__VERIFIER_nondet_pchar(void)` | `map2check_gen_data_fuzzer_pchar` | new | Feature `libfuzzer`. Core target of `map2check_non_det_pchar` (#6). An `unsigned` length, NULL for 0, capped at 4096, then `length - 1` chars and a NUL, as `NonDetGeneratorAFL.c:101-118`; the string comes from the C `malloc` and the program may `free` it (SD-18). | | `__VERIFIER_assume` | `void __VERIFIER_assume(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assume(0)` abandons the current fuzzer input and `LLVMFuzzerTestOneInput` returns 0, as develop's `nondet_cancel`; with no iteration running it calls `exit(0)` (SD-07). Defined `extern "C-unwind"`. The analysed program must be compiled with `-fexceptions` (D1 condition 11); see “`__VERIFIER_assume` under a fuzzer”. | | `map2check_fuzzer_assume` | `void map2check_fuzzer_assume(int32_t condition)` | `map2check_fuzzer_assume(int)` (`nondet_gen_libfuzzer.h:41`) | changed | Feature `libfuzzer`. Same behaviour as `__VERIFIER_assume` (SD-07). | -| `__VERIFIER_error` | `void __VERIFIER_error(void)` | — | new | Feature `libfuzzer`. Records REACHABILITY, prints the JSON report when the mode is unset, then aborts (SD-08). | -| `__VERIFIER_assert` | `void __VERIFIER_assert(int32_t condition)` | — | new | Feature `libfuzzer`. `__VERIFIER_assert(0)` calls `__VERIFIER_error()` (SD-08). | +| `__VERIFIER_error` | `void __VERIFIER_error(void)` | — | new | Features `libfuzzer` and `klee`. With an analysis mode set it does nothing, as develop's; with the mode unset it records REACHABILITY, prints the JSON report and aborts (SD-08). | +| `__VERIFIER_assert` | `void __VERIFIER_assert(int32_t condition)` | — | new | Features `libfuzzer` and `klee`. `__VERIFIER_assert(0)` calls `__VERIFIER_error()`: nothing with a mode set, an abort without (SD-08). | | `next_current_step` | — | `next_current_step()` | discontinued | Use `get_next_step()`; same effect (#13). | | `map2check_save_memtrack_log` | — | `memtracklog.h:44` | discontinued | Covered by the `map2check_map_*` exports; no external caller. | | `map2check_gen_data_fuzzer_*` | — | bool, char, uchar, short, ushort, int32_t, int64_t, uint, unsigned, long, ulong, float, double, size_t | discontinued | Replaced by `__VERIFIER_nondet_*` (#12). | @@ -172,8 +172,8 @@ Map2Check. | `map2check_wasm_check_access` | — | out-of-scope | As above. | | `__VERIFIER_nondet_*` | `int`, `uint`, `char`, `long`, `ushort`, `pointer` | same | Feature `libfuzzer`. | | `__VERIFIER_assume` | `(int)` | same | Feature `libfuzzer` (SD-07). | -| `__VERIFIER_assert` | `(int)` | same | Feature `libfuzzer` (SD-08). | -| `__VERIFIER_error` | `(void)` | same | Feature `libfuzzer` (SD-08). | +| `__VERIFIER_assert` | `(int)` | same | Features `libfuzzer` and `klee` (SD-08). | +| `__VERIFIER_error` | `(void)` | same | Features `libfuzzer` and `klee` (SD-08). | The passes also expect the result channel of the embedded library: the files that the Map2Check frontend parses. This library writes them when a @@ -304,6 +304,31 @@ silently, so Task 3.2 must map each `map2check_nondet_` to its `map2check_save_nondet_log_` exactly as `docs/cutover/abi-mapping.tsv` says. +## KLEE generator (feature `klee`) + +The `symex` runtime (`rust/src/nondet/klee.rs`) is built only from +`klee/Cargo.toml` with rustc 1.72.1 as LLVM 16 bitcode +(`scripts/build-klee-runtime.sh`), not as `libmap2check.a`, and is not in +the header (`cbindgen:ignore`). It exports the same `__VERIFIER_nondet_*`, +`__VERIFIER_assume`, `__VERIFIER_error` and `__VERIFIER_assert` as the +libFuzzer generator, with the same signatures, plus `main`, which calls +`__map2check_main__(argc, argv)` as develop's `NonDetGeneratorKlee.c:16`. +Differences from develop's KLEE generator (`docs/klee-feasibility.md`): + +- each value is a `klee_make_symbolic` object of `sizeof(type)` bytes named + `non_det_` as develop names it; `__VERIFIER_nondet_uint` stands for + develop's `map2check_non_det_uint` and `_unsigned` and is named + `non_det_uint` (the frontend reads both names as a 4-byte unsigned); +- `__VERIFIER_nondet_float` is new (`non_det_float`, 4 bytes), and the + frontend decodes it as a signed int; +- `__VERIFIER_nondet_pchar` caps its length at 4096, as the AFL++ reader, + where develop's KLEE generator has no cap (D1 condition 9); +- `__VERIFIER_assume` is `klee_assume(condition != 0)`; +- `__VERIFIER_error` / `__VERIFIER_assert` behave as in the libFuzzer + generator (SD-08): nothing with a mode set, as develop; +- values are concretised (`klee_get_value_*` + `klee_assume`) before the + witness logs or the JSON report format them (D1 condition 7). + ## `__VERIFIER_assume` under a fuzzer `__VERIFIER_assume(0)` and `map2check_fuzzer_assume(0)` abandon the current diff --git a/docs/klee-feasibility.md b/docs/klee-feasibility.md index 14be3b8..f7c3be9 100644 --- a/docs/klee-feasibility.md +++ b/docs/klee-feasibility.md @@ -40,7 +40,7 @@ O programa já vem dos passes com o `main` renomeado para `__map2check_main__`. | `float` | não existe | 4 bytes, `non_det_float`. O frontend não conhece o nome e o decodifica como int com sinal | | `pchar` | comprimento `non_det_unsigned` sem limite, depois `length - 1` × `non_det_char` | o mesmo, **com teto de 4096** no comprimento, como o leitor AFL++, para que o vetor do KLEE gere a mesma string no replay (condição 9) | | assume | `klee_assume(expr)` | `klee_assume(cond != 0)`. Um assume comprovadamente falso é erro de usuário do KLEE nos dois | -| `__VERIFIER_error` / `_assert` | `__VERIFIER_error` vazio. O alvo vira `map2check_target_function` | registra `Reachability` e aborta, como no gerador libFuzzer (corpo compartilhado em `nondet/mod.rs`) | +| `__VERIFIER_error` / `_assert` | os dois vazios (`Map2CheckFunctions.c:57-59`, `AnalysisModeAssert.c:25-27`). O alvo vira `map2check_target_function`; o assert, `map2check_assert` | com modo definido, **nada**, como em `develop`: o alvo vem só de `map2check_reach_target` e o assert de `map2check_is_valid_assert`. Sem modo (uso direto da API), registra `Reachability` e aborta (SD-08). Corpo compartilhado com o gerador libFuzzer em `nondet/mod.rs` | | Violação | `map2check_exit` → `abort()` | `output/termination.rs` → `std::process::abort()` → `abort` da uClibc. O KLEE registra `.abort.err` nos dois | | Saída | `fprintf` da uClibc com o valor simbólico | `output::prepare_for_output` → `klee::concretize` antes de formatar (condição 7) | @@ -54,7 +54,7 @@ Os comandos rodaram na imagem `map2check-dev:cutover-klee-rust`, construída de DEVELOP_SRC=../Map2Check-cutover tests/klee_paths.sh ``` -Resultado: 16 PASS, 0 FAIL (`all KLEE checks passed`). +Resultado: 18 PASS, 0 FAIL (`all KLEE checks passed`). | Programa | O que verifica | Rust: completos + parciais | `develop` C: completos + parciais | |---|---|---|---| @@ -63,6 +63,8 @@ Resultado: 16 PASS, 0 FAIL (`all KLEE checks passed`). | `range.c` | `x > 1000` atinge o erro. O log formata um valor com muitos dígitos possíveis | **1 + 1** | **1 + 7** | | `json.c` | sem modo, o relatório JSON de um `x` sem restrição (o caso 2 → 22 do spike) | 1 + 0 | não se aplica (o C não tem JSON) | | `deref.c` | leitura fora do bloco com `i == 4`: abort, `FALSE-DEREF`, ktest com `i = 4` | 3 + 1 | 3 + 1 | +| `vererr_deref.c` | modo memtrack: `__VERIFIER_error()` (e, no Rust, `__VERIFIER_assert(0)`) antes da leitura fora do bloco. Com modo definido os dois não fazem nada, e o resultado é `FALSE-DEREF` | 3 + 1 | 3 + 1 | +| `vererr_target.c` | modo target: `__VERIFIER_error()` sozinho não informa nada (2 caminhos, nenhum abort, `NONE`) | 2 + 0 | 2 + 0 | | `layout.c` | oito tipos. O ktest do erro vira semente e passa pelo leitor libFuzzer do Rust | 8 + 1 | 8 + 1 | "Parciais" são os caminhos encerrados por erro, aqui o abort da violação. Exploração completa, sem `--exit-on-error-type`. Os veredictos foram conferidos numa segunda execução com `--exit-on-error-type=Abort`, como o frontend roda os modos de propriedade. @@ -90,13 +92,13 @@ A semente tem esses 30 bytes concatenados. O mesmo `layout.c` foi ligado a `libm ## Achados e limitações -1. **Intrínsecos vetoriais do `std` pré-compilado.** O `std` do 1.72.1 vem vetorizado, e o LTO fat leva esse código para o módulo. O KLEE 3.1 não executa `llvm.vector.reduce.add/or` nem `llvm.x86.sse2.pmulhu.w`. O caminho de `deref.c` morria com `unimplemented intrinsic` em `core::fmt::Formatter::pad_integral`, ao formatar o log de heap em hexadecimal com largura. Desligar a vetorização no LTO (`-C no-vectorize-loops -C no-vectorize-slp`) não resolve, porque o código já vem vetorizado no bitcode do `std`. `scripts/klee-lower-intrinsics.py` troca cada intrínseco por uma função interna escalar (`extractelement`/`insertelement` e aritmética). A equivalência é conferida com `lli-16` em `tests/klee/lower-intrinsics.ll`. Um intrínseco vetorial ou x86 desconhecido faz o build falhar. Isto é uma sexta peça do produto, além das cinco da D1. +1. **Intrínsecos vetoriais do `std` pré-compilado.** O `std` do 1.72.1 vem vetorizado, e o LTO fat leva esse código para o módulo. O KLEE 3.1 não executa `llvm.vector.reduce.add/or` nem `llvm.x86.sse2.pmulhu.w`. O caminho de `deref.c` morria com `unimplemented intrinsic` em `core::fmt::Formatter::pad_integral`, ao formatar o log de heap em hexadecimal com largura. Desligar a vetorização no LTO (`-C no-vectorize-loops -C no-vectorize-slp`) não resolve, porque o código já vem vetorizado no bitcode do `std`. `scripts/klee-lower-intrinsics.py` troca cada intrínseco por uma função interna escalar (`extractelement`/`insertelement` e aritmética). Antes de usar a troca, o build roda dois autotestes: a equivalência com `lli-16` em `tests/klee/lower-intrinsics.ll` e a recusa de `tests/klee/unknown-intrinsic.ll` (`llvm.masked.load`). Depois da troca, qualquer intrínseco declarado que não esteja na lista dos que o KLEE 3.1 executa faz o build falhar. A lista vem dos `case Intrinsic::` de `Executor.cpp` e `IntrinsicCleaner.cpp` na tag v3.1, mais `bitreverse` como exceção justificada (item 3). O build escreve tudo num diretório temporário e só move o `.bc` no fim: uma falha não deixa bitcode no destino. Não usei o `opt-16 -enable-new-pm=0 -expand-reductions`: ele só cobre a família reduce (o script continuaria necessário para os intrínsecos x86) e vive no gerenciador de passes legado que o LLVM está removendo. Isto é uma sexta peça do produto, além das cinco da D1. 2. **`RandomState` do `HashSet`.** O `RandomState::new()` busca a semente do SO por um símbolo fraco (`getrandom`). Sob o KLEE, isso vira `invalid function pointer`, e o caminho morre ao gravar os logs de witness. Os três `HashSet` do crate passaram a usar `memtrack::FixedHashSet`, com chaves SipHash fixas. Eles só respondem pertinência, então o resultado não muda em nenhum build. 3. **`llvm.bitreverse` (miniz_oxide).** Só a impressão de backtrace do `std` chega a ele, e com `panic = "abort"` ela não é executada. O KLEE avisa `unsupported intrinsic` no carregamento. Não dá para trocá-lo: o `--optimize` do KLEE refaz o intrínseco a partir da versão escalar. 4. **Chamadas externas.** O `std` grava os arquivos por `open64`/`write`/`close`/`unlink` externos, como o `fopen`/`fprintf` do C. O KLEE avisa, por exemplo, `undefined reference to function: statx`, `pthread_key_create` e `_Unwind_*`. Nenhum desses símbolos é executado nos caminhos testados. 5. **`--external-calls=all` e o `std`.** O que usa símbolos fracos via `dlsym` ou `syscall!` (entropia, `statx`, TLS com destrutor) é inseguro sob o KLEE. Hoje o crate não usa nada disso no estado, mas qualquer código novo que entre no build `symex` precisa passar por `tests/klee_paths.sh`. 6. **Escopo dos testes.** Os programas imitam a instrumentação dos passes à mão (`tests/klee/m2c.h`). O pipeline real com os passes do Map2Check é a Task 3.x, e o `tests/klee_paths.sh` no CI do Map2Check é a Task 3.4. Overflow, memcleanup, `pchar` e `double` simbólicos não têm programa próprio. `layout.c` cobre os oito tipos inteiros. -7. **CI.** O job `klee_runtime` compila o bitcode em 1.72.1 com `--locked`, confere que ele existe e que o `llvm-dis-16` o lê, roda o `cargo-deny` sobre `klee/Cargo.lock` e publica o `.bc` como artefato. O KLEE não roda no CI desta biblioteca, porque a imagem com o KLEE 3.1 é a de desenvolvimento do Map2Check e não está num registro público. `tests/klee_paths.sh` roda localmente e entra no CI do Map2Check na Task 3.4. +7. **CI.** O job `klee_runtime` roda o clippy 1.72.1 sobre o build `klee` (é o único lint de `nondet/klee.rs`), compila o bitcode em 1.72.1 com `--locked`, com os autotestes da troca de intrínsecos, confere que ele existe e que o `llvm-dis-16` o lê, roda o `cargo-deny` sobre `klee/Cargo.lock` e publica o `.bc` como artefato. O KLEE não roda no CI desta biblioteca, porque a imagem com o KLEE 3.1 é a de desenvolvimento do Map2Check e não está num registro público. `tests/klee_paths.sh` roda localmente e entra no CI do Map2Check na Task 3.4. ## Custos aceitos (D1, "Custo do pin do rustc 1.72.1") From e265bc2be0855451551c30397c4baa52ffc1ba96 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:46:33 -0400 Subject: [PATCH 46/63] feat: port the wasm2c runtime to Rust behind the wasm feature The wasm_rt_* stubs of WasmRuntimeStubs.c, w2c_wasi__snapshot__preview1_proc_exit and map2check_wasm_malloc/free/check_access (AnalysisModeMemtrack.c:385-430), with wasm_rt_memory_t and wasm_rt_funcref_table_t mirrored from wabt 1.0.41. The layout test compares them with wasm-rt.h when WASM_RT_INCLUDE is set. --- Cargo.toml | 3 + rust/src/ffi_tests.rs | 338 ++++++++++++++++++++++ rust/src/lib.rs | 6 +- rust/src/memtrack/mod.rs | 70 +++++ rust/src/state.rs | 3 + rust/src/wasm.rs | 509 +++++++++++++++++++++++++++++++++ tests/ffi_exports_checklist.rs | 12 +- tests/ffi_guard_coverage.rs | 1 + 8 files changed, 940 insertions(+), 2 deletions(-) create mode 100644 rust/src/wasm.rs diff --git a/Cargo.toml b/Cargo.toml index ee80ed6..97bc315 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -23,6 +23,9 @@ tempfile = "3" [features] libfuzzer = [] +# The wasm2c runtime of Map2Check's `--wasm` mode (D2): the `wasm_rt_*` stubs, +# `w2c_wasi__snapshot__preview1_proc_exit` and the `map2check_wasm_*` checks. +wasm = [] # The `klee` feature (the KLEE runtime) lives in klee/Cargo.toml, the # manifest of the rustc 1.72.1 bitcode build (D1, route A'). It is not a # feature here: `--all-features` would link it with `libfuzzer` (both define diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index 482ae12..fbb3797 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -2391,3 +2391,341 @@ fn memtrack_mode_reports_any_leak_reachable_or_not() { store(0x10, 0); assert_eq!(memcleanup_endprog(), P::MemsafetyMemtrack); } + +// ── wasm2c runtime (feature `wasm`, Task 2.8) ─────────────────────────────── +// +// The three `map2check_wasm_*` exports follow `AnalysisModeMemtrack.c:385-430` +// of Map2Check `develop`; the `wasm_rt_*` stubs follow `WasmRuntimeStubs.c`. + +#[cfg(feature = "wasm")] +mod wasm { + use super::*; + use crate::wasm::{ + map2check_wasm_check_access, map2check_wasm_free, map2check_wasm_malloc, + w2c_wasi__snapshot__preview1_proc_exit, wasm_rt_allocate_funcref_table, + wasm_rt_allocate_memory, wasm_rt_free, wasm_rt_free_funcref_table, wasm_rt_free_memory, + wasm_rt_funcref_table_t, wasm_rt_grow_memory, wasm_rt_init, wasm_rt_is_initialized, + wasm_rt_memory_t, + }; + + fn started(mode: Option) { + map2check_reset(); + state::reset_config(); + if let Some(m) = mode { + map2check_set_mode(m as i32); + } + map2check_init(); + } + + fn mem() -> wasm_rt_memory_t { + // SAFETY: an all-zero memory descriptor is what wasm2c hands the + // allocator (null pointers, zero counters). + unsafe { std::mem::zeroed() } + } + + // ── map2check_wasm_malloc ─────────────────────────────────────────────── + + #[test] + fn wasm_malloc_registers_a_block_whose_accesses_are_valid() { + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, 16); + map2check_wasm_check_access(0x1000, 4); + map2check_wasm_check_access(0x100c, 4); + map2check_wasm_check_access(0x100f, 1); + assert_ok(); + } + + #[test] + fn wasm_malloc_ignores_null_and_failed_allocations() { + // AnalysisModeMemtrack.c:386: offset 0 and (uint64_t)-1 are no block. + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0, 16); + map2check_wasm_malloc(u64::MAX, 16); + assert!(snapshot(|s| s.wasm_allocs.is_empty())); + } + + #[test] + fn wasm_malloc_of_a_known_offset_resizes_and_revives_it() { + // AnalysisModeMemtrack.c:387-391: same offset -> size and is_free + // updated in place, no second row. + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, 8); + map2check_wasm_free(0x1000, 3); + map2check_wasm_malloc(0x1000, 32); + assert_eq!(snapshot(|s| s.wasm_allocs.len()), 1); + map2check_wasm_check_access(0x101c, 4); + assert_ok(); + } + + #[test] + fn wasm_malloc_truncates_the_size_to_unsigned_as_develop() { + // `row->size = (unsigned)size` (AnalysisModeMemtrack.c:389). + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, (1u64 << 32) | 8); + assert_eq!(snapshot(|s| s.wasm_allocs[0].size), 8); + } + + // ── map2check_wasm_free ───────────────────────────────────────────────── + + #[test] + fn wasm_free_of_a_tracked_block_marks_it_free_with_size_zero() { + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, 16); + map2check_wasm_free(0x1000, 7); + assert_ok(); + snapshot(|s| { + assert!(s.wasm_allocs[0].is_free); + assert_eq!(s.wasm_allocs[0].size, 0); + }); + } + + #[test] + fn wasm_free_of_an_untracked_offset_is_a_false_free_at_the_line() { + // AnalysisModeMemtrack.c:401: write_property(FALSE_FREE, line, ""). + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_free(0x2000, 41); + assert_violated(ViolatedProperty::MemsafetyFree, 41, ""); + } + + #[test] + fn wasm_double_free_is_not_reported_as_develop() { + // develop only looks the row up; a freed row is still a row. + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, 16); + map2check_wasm_free(0x1000, 7); + map2check_wasm_free(0x1000, 8); + assert_ok(); + } + + // ── map2check_wasm_check_access ───────────────────────────────────────── + + #[test] + fn wasm_access_past_the_block_is_reported_as_develop_does() { + // AnalysisModeMemtrack.c:420-423: FALSE_OVERFLOW, line 0, no function + // (the report tag is OVERFLOW; see SD-wasm-access in semantic-deltas). + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, 16); + map2check_wasm_check_access(0x1010, 4); + assert_violated(ViolatedProperty::Overflow, 0, ""); + } + + #[test] + fn wasm_access_straddling_the_end_is_invalid() { + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, 16); + map2check_wasm_check_access(0x100d, 4); + assert_violated(ViolatedProperty::Overflow, 0, ""); + } + + #[test] + fn wasm_access_below_the_block_is_invalid() { + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, 16); + map2check_wasm_check_access(0xfff, 1); + assert_violated(ViolatedProperty::Overflow, 0, ""); + } + + #[test] + fn wasm_access_after_free_is_invalid() { + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, 16); + map2check_wasm_free(0x1000, 5); + map2check_wasm_check_access(0x1000, 4); + assert_violated(ViolatedProperty::Overflow, 0, ""); + } + + #[test] + fn wasm_access_with_no_block_is_invalid() { + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_check_access(0x1000, 1); + assert_violated(ViolatedProperty::Overflow, 0, ""); + } + + #[test] + fn wasm_access_inside_a_freed_inner_block_is_covered_by_the_older_block() { + // A freed row keeps size 0, so its range + // `[offset, offset + 0 - access + 1)` is empty and the scan goes on + // to the older row that still covers the address + // (AllocationLog.c:127-148). Develop's behaviour, kept as is. + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, 64); + map2check_wasm_malloc(0x1010, 16); + map2check_wasm_free(0x1010, 5); + map2check_wasm_check_access(0x1014, 4); + assert_ok(); + } + + #[test] + fn wasm_first_violation_is_final() { + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_free(0x2000, 41); + map2check_wasm_check_access(0x1000, 1); + assert_violated(ViolatedProperty::MemsafetyFree, 41, ""); + } + + // ── mode gating ───────────────────────────────────────────────────────── + + #[test] + fn wasm_checks_are_active_with_the_mode_unset() { + let _guard = fresh(); + started(None); + map2check_wasm_check_access(0x1000, 1); + assert_violated(ViolatedProperty::Overflow, 0, ""); + } + + #[test] + fn wasm_checks_are_inactive_in_modes_without_memory_checks() { + for mode in [ + AnalysisMode::None, + AnalysisMode::Memcleanup, + AnalysisMode::Overflow, + AnalysisMode::Assert, + AnalysisMode::Target, + ] { + let _guard = fresh(); + started(Some(mode)); + map2check_wasm_free(0x2000, 41); + map2check_wasm_check_access(0x1000, 1); + assert_ok(); + } + } + + #[test] + fn wasm_tracking_records_in_every_mode() { + let _guard = fresh(); + started(Some(AnalysisMode::Memcleanup)); + map2check_wasm_malloc(0x1000, 16); + assert_eq!(snapshot(|s| s.wasm_allocs.len()), 1); + } + + #[test] + fn wasm_table_is_cleared_by_reset_and_init() { + let _guard = fresh(); + started(Some(AnalysisMode::Memtrack)); + map2check_wasm_malloc(0x1000, 16); + map2check_init(); + assert!(snapshot(|s| s.wasm_allocs.is_empty())); + map2check_wasm_malloc(0x1000, 16); + map2check_reset(); + assert!(snapshot(|s| s.wasm_allocs.is_empty())); + } + + // ── wasm_rt_* stubs (WasmRuntimeStubs.c) ──────────────────────────────── + + #[test] + fn rt_allocate_memory_zeroes_and_sets_every_field() { + let mut m = mem(); + // SAFETY: `m` is a valid descriptor; freed below. + unsafe { wasm_rt_allocate_memory(&mut m, 2, 5, false, 65536) }; + assert!(!m.data.is_null()); + assert_eq!(m.size, 2 * 65536); + assert_eq!(m.pages, 2); + assert_eq!(m.max_pages, 5); + assert_eq!(m.page_size, 65536); + // SAFETY: data_end points one past the allocation that data starts. + assert_eq!(unsafe { m.data.add(m.size as usize) }, m.data_end); + // SAFETY: the block is `size` bytes long and calloc'd. + let all_zero = (0..m.size as usize).all(|i| unsafe { *m.data.add(i) } == 0); + assert!(all_zero); + // SAFETY: `m` was allocated above. + unsafe { wasm_rt_free_memory(&mut m) }; + assert!(m.data.is_null()); + assert_eq!(m.size, 0); + } + + #[test] + fn rt_grow_memory_returns_old_pages_and_zero_fills() { + let mut m = mem(); + // SAFETY: valid descriptor, freed below. + unsafe { + wasm_rt_allocate_memory(&mut m, 1, 3, false, 65536); + *m.data = 0xAA; + assert_eq!(wasm_rt_grow_memory(&mut m, 0), 1); + assert_eq!(wasm_rt_grow_memory(&mut m, 1), 1); + } + assert_eq!(m.pages, 2); + assert_eq!(m.size, 2 * 65536); + // SAFETY: the block now has 2 pages. + unsafe { + assert_eq!(*m.data, 0xAA, "old contents are kept"); + assert_eq!(*m.data.add(65536), 0, "new pages are zeroed"); + assert_eq!(*m.data.add(2 * 65536 - 1), 0); + assert_eq!(m.data.add(m.size as usize), m.data_end); + wasm_rt_free_memory(&mut m); + } + } + + #[test] + fn rt_grow_memory_past_the_maximum_fails_with_uint32_max() { + let mut m = mem(); + // SAFETY: valid descriptor, freed below. + unsafe { + wasm_rt_allocate_memory(&mut m, 1, 2, false, 65536); + assert_eq!(wasm_rt_grow_memory(&mut m, 2), u64::from(u32::MAX)); + assert_eq!(wasm_rt_grow_memory(&mut m, u64::MAX), u64::from(u32::MAX)); + wasm_rt_free_memory(&mut m); + } + assert_eq!(m.pages, 1); + } + + #[test] + fn rt_grow_memory_defaults_a_zero_page_size_to_64k() { + let mut m = mem(); + // SAFETY: valid descriptor, freed below. + unsafe { + wasm_rt_allocate_memory(&mut m, 1, 4, false, 65536); + m.page_size = 0; + assert_eq!(wasm_rt_grow_memory(&mut m, 1), 1); + assert_eq!(m.size, 2 * 65536); + wasm_rt_free_memory(&mut m); + } + } + + #[test] + fn rt_funcref_table_allocates_zeroed_elements() { + // SAFETY: an all-zero table descriptor is valid. + let mut t: wasm_rt_funcref_table_t = unsafe { std::mem::zeroed() }; + // SAFETY: valid descriptor, freed below. + unsafe { wasm_rt_allocate_funcref_table(&mut t, 4, 10) }; + assert!(!t.data.is_null()); + assert_eq!(t.size, 4); + // SAFETY: 4 funcref elements, calloc'd, so the last field of the last + // one is addressable. + unsafe { + assert!((*t.data.add(3)).module_instance.is_null()); + wasm_rt_free_funcref_table(&mut t); + } + assert!(t.data.is_null()); + } + + #[test] + fn rt_init_free_and_is_initialized_follow_the_stub() { + // The stub is "always initialised for KLEE": true from the start, + // false after wasm_rt_free, true again after wasm_rt_init. + assert!(wasm_rt_is_initialized()); + wasm_rt_free(); + assert!(!wasm_rt_is_initialized()); + wasm_rt_init(); + assert!(wasm_rt_is_initialized()); + } + + #[test] + fn rt_proc_exit_is_a_no_op() { + // SAFETY: the instance is unused. + unsafe { w2c_wasi__snapshot__preview1_proc_exit(std::ptr::null_mut(), 7) }; + } +} diff --git a/rust/src/lib.rs b/rust/src/lib.rs index a5c0bbe..8c6cc7f 100644 --- a/rust/src/lib.rs +++ b/rust/src/lib.rs @@ -3,7 +3,8 @@ // map2check-library — Rust rewrite of the C map2check-library. // // Unsafe code policy: -// - `ffi.rs` is the sole module containing unsafe code (FFI boundary). +// - `ffi.rs` is the sole module containing unsafe code (FFI boundary), with +// `wasm.rs` (feature `wasm`: libc allocation and wasm-rt struct access). // - All other modules use `#![deny(unsafe_code)]` internally. // - Every unsafe block in `ffi.rs` is annotated with a SAFETY comment. // @@ -26,3 +27,6 @@ pub mod memtrack; pub mod nondet; pub mod output; pub mod state; +/// wasm2c runtime (`--wasm` mode of Map2Check) — feature-gated. +#[cfg(feature = "wasm")] +pub mod wasm; diff --git a/rust/src/memtrack/mod.rs b/rust/src/memtrack/mod.rs index 6c4d15c..ea5ec01 100644 --- a/rust/src/memtrack/mod.rs +++ b/rust/src/memtrack/mod.rs @@ -13,6 +13,76 @@ use std::hash::BuildHasherDefault; /// questions, so the hash seed changes nothing in their results. pub(crate) type FixedHashSet = HashSet>; +/// One row of develop's allocation log as the wasm2c runtime uses it +/// (`MEMORY_ALLOCATIONS_ROW`, `Map2CheckTypes.h:113-121`): the linear-memory +/// offset the lifted `malloc` returned, the block size and whether it was +/// freed. Offsets are wasm linear-memory offsets, not host addresses. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct WasmAlloc { + pub offset: u64, + /// `unsigned` in develop: the size is truncated to 32 bits. + pub size: u32, + pub is_free: bool, +} + +/// `map2check_wasm_malloc` (`AnalysisModeMemtrack.c:385-397`): offset 0 and +/// `(uint64_t)-1` are no block; a known offset (newest row first, +/// `find_row_with_address`, `AllocationLog.c:103-118`) is resized and revived +/// in place; any other offset gets a new row. +pub fn wasm_register_malloc(table: &mut Vec, offset: u64, size: u64) { + if offset == 0 || offset == u64::MAX { + return; + } + let size = size as u32; + match table.iter_mut().rev().find(|row| row.offset == offset) { + Some(row) => { + row.size = size; + row.is_free = false; + } + None => table.push(WasmAlloc { + offset, + size, + is_free: false, + }), + } +} + +/// `map2check_wasm_free` (`AnalysisModeMemtrack.c:399-407`): marks the block +/// at `offset` free with size 0 and returns true; false when no row has that +/// offset. A row that is already free is found again, so a double free is not +/// an error, as in develop. +pub fn wasm_register_free(table: &mut [WasmAlloc], offset: u64) -> bool { + match table.iter_mut().rev().find(|row| row.offset == offset) { + Some(row) => { + row.is_free = true; + row.size = 0; + true + } + None => false, + } +} + +/// `is_valid_allocation_address` (`AllocationLog.c:127-148`) for +/// `map2check_wasm_check_access`: the newest row whose range +/// `[offset, offset + size - access + 1)` holds `offset` decides, and it is +/// valid only if the block is live. The arithmetic is develop's: the access +/// size is cut to `int` (`(int)access_size`) and sign-extended, and the sum +/// wraps at 64 bits. +pub fn wasm_is_valid_access(table: &[WasmAlloc], address: u64, access_size: u64) -> bool { + let access = i64::from(access_size as i32) as u64; + for row in table.iter().rev() { + let top = row + .offset + .wrapping_add(u64::from(row.size)) + .wrapping_sub(access) + .wrapping_add(1); + if row.offset <= address && address < top { + return !row.is_free; + } + } + false +} + /// Single memory tracking record — analogous to `memtrack_log_t` in C. /// Addresses are `usize` (platform pointer width) instead of `long`. #[derive(Debug, Clone, Serialize)] diff --git a/rust/src/state.rs b/rust/src/state.rs index 3529e22..81420c8 100644 --- a/rust/src/state.rs +++ b/rust/src/state.rs @@ -80,6 +80,9 @@ pub struct AnalysisState { pub result: AnalysisResult, pub nondets: Vec, pub memtrack: Vec, + /// The offset-keyed allocation table of the wasm2c runtime + /// (`map2check_wasm_*`, Task 2.8). + pub wasm_allocs: Vec, pub bbtrack: Vec, pub legacy_nondet_logs: Vec>, pub legacy_bbtrack_logs: Vec>, diff --git a/rust/src/wasm.rs b/rust/src/wasm.rs new file mode 100644 index 0000000..9dd8e3f --- /dev/null +++ b/rust/src/wasm.rs @@ -0,0 +1,509 @@ +// SPDX-License-Identifier: GPL-3.0-only +// +// The runtime of Map2Check's `--wasm` mode (decision D2). Map2Check lifts a +// WebAssembly binary to C with wabt's wasm2c, instruments the lifted code and +// links `WasmRuntimeStubs.bc` for the `wasm_rt_*` functions the lifted code +// calls. This module is that support, ported from the C runtime of Map2Check +// `develop`: +// +// - `rt`: `WasmRuntimeStubs.c` (calloc/realloc based memory, KLEE-friendly), +// with `wasm_rt_memory_t` and `wasm_rt_funcref_table_t` mirrored from +// wabt 1.0.41's `wasm-rt.h` (layout checked by the `layout` test against +// the real header, see below); +// - `map2check_wasm_malloc`, `map2check_wasm_free` and +// `map2check_wasm_check_access`: `AnalysisModeMemtrack.c:385-430`, the +// per-allocation bounds check of lifted code, over the offset-keyed table +// in `AnalysisState::wasm_allocs` (`memtrack::wasm_*`). +// +// Unsafe code is confined to this file (raw wasm-rt structs and libc +// allocation) and `ffi.rs`. + +use crate::{analysismode::mode::Check, caller::ViolatedProperty, ffi::ffi_guard, memtrack, state}; + +/// Whether `check` is active in the selected analysis mode. +fn check_active(check: Check) -> bool { + crate::analysismode::mode::check_enabled(state::config().mode, check) +} + +/// Records a violation and, with a mode set, ends the run (the same path as +/// every other check: first violation final, abort in a Map2Check run). +fn record(property: ViolatedProperty, line: u32) { + match state::with_state(|s| { + let first = s.result.ok; + s.set_false(property, line, ""); + Ok(first) + }) { + Ok(true) => crate::output::termination::on_violation(false), + Ok(false) => {} + Err(e) => eprintln!("[map2check] error: {e}"), + } +} + +// ── The memory checks of lifted code ──────────────────────────────────────── + +/// Registers the block that the lifted `malloc` returned at linear-memory +/// `offset` (`map2check_wasm_malloc`, `AnalysisModeMemtrack.c:385-397`). The +/// pass emits the call right after the call to `w2c_*_malloc`, with the +/// zero-extended result and size. Offsets 0 and `(uint64_t)-1` are ignored. +/// Tracking happens in every mode. +#[no_mangle] +pub extern "C" fn map2check_wasm_malloc(offset: u64, size: u64) { + ffi_guard( + || { + if let Err(e) = state::with_state(|s| { + memtrack::wasm_register_malloc(&mut s.wasm_allocs, offset, size); + Ok(()) + }) { + eprintln!("[map2check] error: {e}"); + } + }, + (), + ); +} + +/// Releases the block at linear-memory `offset` (`map2check_wasm_free`, +/// `AnalysisModeMemtrack.c:399-407`). An offset that was never registered is +/// a `MEMSAFETY_FREE` violation at `line` (the pass sends the source line as +/// an `i64`); an already freed block is found again and is not an error. +/// The violation is reported only where the memory checks are active. +#[no_mangle] +pub extern "C" fn map2check_wasm_free(offset: u64, line: i64) { + ffi_guard( + || { + let found = match state::with_state(|s| { + Ok(memtrack::wasm_register_free(&mut s.wasm_allocs, offset)) + }) { + Ok(found) => found, + Err(e) => { + eprintln!("[map2check] error: {e}"); + return; + } + }; + if !found && check_active(Check::Free) { + // develop passes `int line`: the low 32 bits. + record(ViolatedProperty::MemsafetyFree, line as i32 as u32); + } + }, + (), + ); +} + +/// Checks that `access_size` bytes at linear-memory `offset` lie inside a +/// live block (`map2check_wasm_check_access`, +/// `AnalysisModeMemtrack.c:416-423`). The pass emits it before every load and +/// store through the lifted memory. A bad access is reported as develop does, +/// `FALSE_OVERFLOW` with line 0 and no function name: the report tag is +/// `OVERFLOW`. Reported only where the memory checks are active. +#[no_mangle] +pub extern "C" fn map2check_wasm_check_access(offset: u64, access_size: u64) { + ffi_guard( + || { + if !check_active(Check::Deref) { + return; + } + let valid = match state::with_state(|s| { + Ok(memtrack::wasm_is_valid_access( + &s.wasm_allocs, + offset, + access_size, + )) + }) { + Ok(valid) => valid, + Err(e) => { + eprintln!("[map2check] error: {e}"); + return; + } + }; + if !valid { + record(ViolatedProperty::Overflow, 0); + } + }, + (), + ); +} + +// ── `WasmRuntimeStubs.c` ──────────────────────────────────────────────────── + +/// The `wasm_rt_*` functions the lifted code calls. Not part of the Map2Check +/// C header (`wasm-rt.h` declares them), so cbindgen skips this module. +/// +/// cbindgen:ignore +#[allow(non_camel_case_types)] +pub mod rt { + use std::os::raw::{c_char, c_void}; + use std::sync::atomic::{AtomicBool, Ordering}; + + use crate::ffi::ffi_guard; + + extern "C" { + fn calloc(count: usize, size: usize) -> *mut c_void; + fn realloc(ptr: *mut c_void, size: usize) -> *mut c_void; + fn free(ptr: *mut c_void); + } + + /// `wasm_rt_memory_t` of wabt 1.0.41 (`wasm-rt.h:466-482`). + #[repr(C)] + #[derive(Debug, Clone, Copy)] + pub struct wasm_rt_memory_t { + pub data: *mut u8, + pub data_end: *mut u8, + pub page_size: u32, + pub pages: u64, + pub max_pages: u64, + pub size: u64, + pub is64: bool, + } + + /// `wasm_rt_tailcallee_t` (`wasm-rt.h:421-425`). + #[repr(C)] + #[derive(Debug, Clone, Copy)] + pub struct wasm_rt_tailcallee_t { + pub func: + Option, + } + + /// `wasm_rt_funcref_t` (`wasm-rt.h:437-453`). + #[repr(C)] + #[derive(Debug, Clone, Copy)] + pub struct wasm_rt_funcref_t { + pub func_type: *const c_char, + pub func: Option, + pub func_tailcallee: wasm_rt_tailcallee_t, + pub module_instance: *mut c_void, + } + + /// `wasm_rt_funcref_table_t` (`wasm-rt.h:519-529`). + #[repr(C)] + #[derive(Debug, Clone, Copy)] + pub struct wasm_rt_funcref_table_t { + pub data: *mut wasm_rt_funcref_t, + pub max_size: u32, + pub size: u32, + } + + /// Allocates linear memory with `calloc` (the default wasm-rt uses + /// `mmap`, which KLEE models poorly). As in develop, `is64` and the + /// `is64` field are not touched. + /// + /// # Safety + /// `mem` must point to a writable `wasm_rt_memory_t`. + #[no_mangle] + pub unsafe extern "C" fn wasm_rt_allocate_memory( + mem: *mut wasm_rt_memory_t, + initial_pages: u64, + max_pages: u64, + _is64: bool, + page_size: u32, + ) { + ffi_guard( + || { + let byte_length = u64::from(page_size).wrapping_mul(initial_pages); + // SAFETY: calloc has no preconditions; the caller guarantees `mem`. + let data = unsafe { calloc(1, byte_length as usize) }.cast::(); + // SAFETY: caller guarantees `mem` is a valid, writable descriptor. + let mem = unsafe { &mut *mem }; + mem.data = data; + mem.data_end = data.wrapping_add(byte_length as usize); + mem.page_size = page_size; + mem.size = byte_length; + mem.pages = initial_pages; + mem.max_pages = max_pages; + }, + (), + ); + } + + /// Grows linear memory by `pages` with `realloc`. Returns the previous + /// page count, or `UINT32_MAX` on failure, as `wasm-rt.h` documents. + /// + /// # Safety + /// `mem` must point to a memory set up by [`wasm_rt_allocate_memory`]. + #[no_mangle] + pub unsafe extern "C" fn wasm_rt_grow_memory(mem: *mut wasm_rt_memory_t, pages: u64) -> u64 { + ffi_guard( + || { + let failure = u64::from(u32::MAX); + // SAFETY: caller guarantees `mem` is a valid descriptor. + let mem = unsafe { &mut *mem }; + let old_pages = mem.pages; + let new_pages = old_pages.wrapping_add(pages); + if new_pages < old_pages || new_pages > mem.max_pages { + return failure; + } + if pages == 0 { + return old_pages; + } + let page_size = if mem.page_size != 0 { + mem.page_size + } else { + 65536 + }; + let old_size = mem.size; + let new_size = new_pages.wrapping_mul(u64::from(page_size)); + // SAFETY: `mem.data` came from calloc/realloc (or is null). + let new_data = + unsafe { realloc(mem.data.cast::(), new_size as usize) }.cast::(); + if new_data.is_null() { + return failure; + } + if new_size > old_size { + // SAFETY: the block now holds `new_size` bytes and the + // range starts at `old_size`, inside it. + unsafe { + new_data + .add(old_size as usize) + .write_bytes(0, (new_size - old_size) as usize); + } + } + mem.data = new_data; + mem.data_end = new_data.wrapping_add(new_size as usize); + mem.pages = new_pages; + mem.size = new_size; + old_pages + }, + u64::from(u32::MAX), + ) + } + + /// Frees linear memory. + /// + /// # Safety + /// `mem` must point to a memory set up by [`wasm_rt_allocate_memory`]. + #[no_mangle] + pub unsafe extern "C" fn wasm_rt_free_memory(mem: *mut wasm_rt_memory_t) { + ffi_guard( + || { + // SAFETY: caller guarantees `mem` is a valid descriptor whose + // data came from calloc/realloc. + unsafe { + free((*mem).data.cast::()); + (*mem).data = std::ptr::null_mut(); + (*mem).size = 0; + } + }, + (), + ); + } + + /// Allocates a funcref table of `elements` null entries. Unlike develop's + /// stub, which sizes each element as `sizeof(void*)` although a + /// `wasm_rt_funcref_t` is four words (a heap overflow once the module's + /// element segments are written), each element gets its real size. + /// `max_size` is left untouched, as in develop. + /// + /// # Safety + /// `table` must point to a writable `wasm_rt_funcref_table_t`. + #[no_mangle] + pub unsafe extern "C" fn wasm_rt_allocate_funcref_table( + table: *mut wasm_rt_funcref_table_t, + elements: u32, + _max_elements: u32, + ) { + ffi_guard( + || { + // SAFETY: calloc has no preconditions. + let data = + unsafe { calloc(elements as usize, std::mem::size_of::()) } + .cast::(); + // SAFETY: caller guarantees `table` is valid and writable. + unsafe { + (*table).data = data; + (*table).size = elements; + } + }, + (), + ); + } + + /// Frees a funcref table. + /// + /// # Safety + /// `table` must point to a table set up by + /// [`wasm_rt_allocate_funcref_table`]. + #[no_mangle] + pub unsafe extern "C" fn wasm_rt_free_funcref_table(table: *mut wasm_rt_funcref_table_t) { + ffi_guard( + || { + // SAFETY: caller guarantees `table` is valid; data is calloc'd. + unsafe { + free((*table).data.cast::()); + (*table).data = std::ptr::null_mut(); + } + }, + (), + ); + } + + /// WASI `proc_exit`: a no-op, so that the analysis, not the lifted + /// program, decides when the run ends. + /// + /// # Safety + /// `instance` is never dereferenced and may be null. + #[no_mangle] + pub unsafe extern "C" fn w2c_wasi__snapshot__preview1_proc_exit( + instance: *mut c_void, + code: u32, + ) { + ffi_guard( + || { + let _ = (instance, code); + }, + (), + ); + } + + /// "Always initialised for KLEE": true from the start. + static INITIALIZED: AtomicBool = AtomicBool::new(true); + + #[no_mangle] + pub extern "C" fn wasm_rt_is_initialized() -> bool { + ffi_guard(|| INITIALIZED.load(Ordering::Relaxed), true) + } + + #[no_mangle] + pub extern "C" fn wasm_rt_init() { + ffi_guard(|| INITIALIZED.store(true, Ordering::Relaxed), ()); + } + + #[no_mangle] + pub extern "C" fn wasm_rt_free() { + ffi_guard(|| INITIALIZED.store(false, Ordering::Relaxed), ()); + } + + /// A trap ends the run. develop calls `map2check_error()`, which writes no + /// property and aborts, so the frontend reads UNKNOWN; here the run + /// records an UNKNOWN verdict (a first violation, final) and aborts. + /// `wasm-rt.h` declares the function `noreturn`, so it never returns, not + /// even with no analysis mode set. + #[no_mangle] + pub extern "C" fn wasm_rt_trap(code: u32) -> ! { + ffi_guard( + || { + let _ = code; + super::record(crate::caller::ViolatedProperty::Unknown, 0); + }, + (), + ); + std::process::abort() + } +} + +pub use rt::*; + +#[cfg(test)] +mod tests { + use super::rt::*; + use std::mem::{offset_of, size_of}; + use std::process::Command; + + /// Compares the Rust mirrors with the layout the C compiler gives + /// `wasm-rt.h`. Runs only when `WASM_RT_INCLUDE` names the directory of + /// the header (wabt 1.0.41: `/opt/wabt-1.0.41/include`); the library + /// itself never needs `cc`. The probe is built in a temporary directory + /// and is not linked into the library. + #[test] + fn layout_matches_wasm_rt_h() { + let Some(include) = std::env::var_os("WASM_RT_INCLUDE") else { + eprintln!( + "layout_matches_wasm_rt_h: SKIPPED, WASM_RT_INCLUDE is not set \ + (set it to the directory of wabt 1.0.41's wasm-rt.h to run it)" + ); + return; + }; + let dir = tempfile::tempdir().unwrap(); + let src = dir.path().join("probe.c"); + std::fs::write( + &src, + r#" +#include +#include +#include "wasm-rt.h" +#define P(name, value) printf("%s %zu\n", name, (size_t)(value)) +int main(void) { + P("memory.size", sizeof(wasm_rt_memory_t)); + P("memory.data", offsetof(wasm_rt_memory_t, data)); + P("memory.data_end", offsetof(wasm_rt_memory_t, data_end)); + P("memory.page_size", offsetof(wasm_rt_memory_t, page_size)); + P("memory.pages", offsetof(wasm_rt_memory_t, pages)); + P("memory.max_pages", offsetof(wasm_rt_memory_t, max_pages)); + P("memory.size_field", offsetof(wasm_rt_memory_t, size)); + P("memory.is64", offsetof(wasm_rt_memory_t, is64)); + P("funcref.size", sizeof(wasm_rt_funcref_t)); + P("funcref.func_type", offsetof(wasm_rt_funcref_t, func_type)); + P("funcref.func", offsetof(wasm_rt_funcref_t, func)); + P("funcref.func_tailcallee", offsetof(wasm_rt_funcref_t, func_tailcallee)); + P("funcref.module_instance", offsetof(wasm_rt_funcref_t, module_instance)); + P("table.size", sizeof(wasm_rt_funcref_table_t)); + P("table.data", offsetof(wasm_rt_funcref_table_t, data)); + P("table.max_size", offsetof(wasm_rt_funcref_table_t, max_size)); + P("table.size_field", offsetof(wasm_rt_funcref_table_t, size)); + P("trap_t.size", sizeof(wasm_rt_trap_t)); + return 0; +} +"#, + ) + .unwrap(); + let exe = dir.path().join("probe"); + let cc = std::env::var("CC").unwrap_or_else(|_| "cc".to_string()); + let built = Command::new(cc) + .arg("-I") + .arg(&include) + .arg(&src) + .arg("-o") + .arg(&exe) + .output() + .expect("cc must be installed to run the layout test"); + assert!( + built.status.success(), + "the probe does not compile: {}", + String::from_utf8_lossy(&built.stderr) + ); + let run = Command::new(&exe).output().unwrap(); + assert!(run.status.success()); + let text = String::from_utf8(run.stdout).unwrap(); + let get = |name: &str| -> usize { + text.lines() + .find_map(|l| l.strip_prefix(name)?.strip_prefix(' ')?.parse().ok()) + .unwrap_or_else(|| panic!("probe printed no `{name}`: {text}")) + }; + let expected = [ + ("memory.size", size_of::()), + ("memory.data", offset_of!(wasm_rt_memory_t, data)), + ("memory.data_end", offset_of!(wasm_rt_memory_t, data_end)), + ("memory.page_size", offset_of!(wasm_rt_memory_t, page_size)), + ("memory.pages", offset_of!(wasm_rt_memory_t, pages)), + ("memory.max_pages", offset_of!(wasm_rt_memory_t, max_pages)), + ("memory.size_field", offset_of!(wasm_rt_memory_t, size)), + ("memory.is64", offset_of!(wasm_rt_memory_t, is64)), + ("funcref.size", size_of::()), + ( + "funcref.func_type", + offset_of!(wasm_rt_funcref_t, func_type), + ), + ("funcref.func", offset_of!(wasm_rt_funcref_t, func)), + ( + "funcref.func_tailcallee", + offset_of!(wasm_rt_funcref_t, func_tailcallee), + ), + ( + "funcref.module_instance", + offset_of!(wasm_rt_funcref_t, module_instance), + ), + ("table.size", size_of::()), + ("table.data", offset_of!(wasm_rt_funcref_table_t, data)), + ( + "table.max_size", + offset_of!(wasm_rt_funcref_table_t, max_size), + ), + ( + "table.size_field", + offset_of!(wasm_rt_funcref_table_t, size), + ), + // `wasm_rt_trap` takes the enum as a u32. + ("trap_t.size", size_of::()), + ]; + for (name, rust) in expected { + assert_eq!(rust, get(name), "layout of `{name}` differs from wasm-rt.h"); + } + } +} diff --git a/tests/ffi_exports_checklist.rs b/tests/ffi_exports_checklist.rs index f3f463e..a5802c1 100644 --- a/tests/ffi_exports_checklist.rs +++ b/tests/ffi_exports_checklist.rs @@ -307,6 +307,13 @@ plain!(not_an_export); /// bitcode, não pelo header. const KLEE_RUNTIME: &str = "rust/src/nondet/klee.rs"; +/// Os `wasm_rt_*` e o `w2c_wasi__*` de `rust/src/wasm.rs` (feature `wasm`) +/// implementam funções que o `wasm-rt.h` do wabt declara: ficam fora do +/// header (`cbindgen:ignore`), e `core_api_coverage` os confere no staticlib. +fn is_wasm_rt_export(name: &str) -> bool { + name.starts_with("wasm_rt_") || name.starts_with("w2c_wasi__") +} + #[test] fn todo_export_do_codigo_esta_no_header() { let rust_src: String = rust_sources() @@ -315,7 +322,10 @@ fn todo_export_do_codigo_esta_no_header() { .map(|p| read(p)) .collect::>() .join("\n"); - let in_source = source_exports(&rust_src); + let in_source: BTreeSet = source_exports(&rust_src) + .into_iter() + .filter(|n| !is_wasm_rt_export(n)) + .collect(); let in_header = header_functions(&read("include/map2check.h")); let not_in_header: Vec<_> = in_source.difference(&in_header).collect(); let not_in_source: Vec<_> = in_header.difference(&in_source).collect(); diff --git a/tests/ffi_guard_coverage.rs b/tests/ffi_guard_coverage.rs index 4194353..e409d2b 100644 --- a/tests/ffi_guard_coverage.rs +++ b/tests/ffi_guard_coverage.rs @@ -11,6 +11,7 @@ const SOURCES: &[&str] = &[ "rust/src/ffi.rs", "rust/src/nondet/libfuzzer.rs", "rust/src/nondet/klee.rs", + "rust/src/wasm.rs", ]; /// Drops comments and string/char-literal contents so braces inside them do From e10481448fb71ec7378e89781bdf2fc4d76c2c68 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:46:33 -0400 Subject: [PATCH 47/63] test: require every core_symbol of the ABI mapping in the runtime staticlib --- tests/core_api_coverage.rs | 85 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 tests/core_api_coverage.rs diff --git a/tests/core_api_coverage.rs b/tests/core_api_coverage.rs new file mode 100644 index 0000000..5b174e8 --- /dev/null +++ b/tests/core_api_coverage.rs @@ -0,0 +1,85 @@ +// SPDX-License-Identifier: GPL-3.0-only +// Todo símbolo núcleo usado pelo mapeamento ABI C → núcleo precisa existir no +// staticlib do perfil `runtime` com as features do runtime (`libfuzzer,wasm`). +// Não há camada de compatibilidade (D5): nomes do runtime C que o núcleo não +// usa não podem aparecer no artefato. + +/// Nomes do runtime C que o núcleo também exporta, com assinatura própria, sem +/// ser alvo de nenhuma linha do mapeamento: `map2check_check_deref` do núcleo é +/// `(ptr, scope, line, fn)` e a linha do mapeamento (`core_symbol` = `-`) diz +/// que o passe deixa de emitir a chamada do C, `(i64 line, ptr fn)`. +const CORE_NAMES_SHARED_WITH_C: &[&str] = &["map2check_check_deref"]; + +fn target_dir(root: &str) -> String { + std::env::var("CARGO_TARGET_DIR").unwrap_or_else(|_| format!("{root}/target")) +} + +fn defined_symbols() -> String { + let root = env!("CARGO_MANIFEST_DIR"); + let status = + std::process::Command::new(std::env::var("CARGO").unwrap_or_else(|_| "cargo".into())) + .args([ + "build", + "--profile", + "runtime", + "--features", + "libfuzzer,wasm", + ]) + .current_dir(root) + .status() + .expect("cargo build"); + assert!(status.success(), "cargo build --profile runtime falhou"); + let archive = format!("{}/runtime/libmap2check.a", target_dir(root)); + let nm = std::process::Command::new("nm") + .args(["--defined-only", "-g", &archive]) + .output() + .expect("nm"); + assert!(nm.status.success(), "nm {archive} falhou"); + String::from_utf8_lossy(&nm.stdout).into_owned() +} + +fn mapping() -> Vec<(String, String)> { + let root = env!("CARGO_MANIFEST_DIR"); + std::fs::read_to_string(format!("{root}/docs/cutover/abi-mapping.tsv")) + .unwrap() + .lines() + .skip(1) + .map(|l| { + let mut c = l.split('\t'); + (c.next().unwrap().into(), c.next().unwrap().into()) + }) + .collect() +} + +fn is_defined(defined: &str, sym: &str) -> bool { + defined.lines().any(|d| d.ends_with(&format!(" T {sym}"))) +} + +#[test] +fn every_core_target_is_exported() { + let defined = defined_symbols(); + let missing: Vec<_> = mapping() + .iter() + .flat_map(|(_, core)| core.split('+').map(str::to_owned).collect::>()) + .filter(|c| c != "-" && c != "__map2check_main__" && !is_defined(&defined, c)) + .collect(); + assert!(missing.is_empty(), "alvos núcleo ausentes: {missing:?}"); +} + +#[test] +fn no_legacy_c_name_is_exported() { + let defined = defined_symbols(); + let leaked: Vec<_> = mapping() + .into_iter() + .filter(|(c, core)| { + !core.split('+').any(|x| x == c) + && !CORE_NAMES_SHARED_WITH_C.contains(&c.as_str()) + && is_defined(&defined, c) + }) + .map(|(c, _)| c) + .collect(); + assert!( + leaked.is_empty(), + "nomes do runtime C exportados (D5 proíbe): {leaked:?}" + ); +} From 3ad91d9ac9e8e6f37420d6a63e53d4888952d3b0 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 10:46:33 -0400 Subject: [PATCH 48/63] docs: document the wasm feature, the wasm exports and SD-19; check wasm-rt.h layout in CI --- .github/workflows/CI.yml | 10 ++++++++ CHANGELOG.md | 5 ++++ docs/api-deltas.md | 35 ++++++++++++++++++++++--- docs/cutover/abi-mapping.tsv | 6 ++--- docs/semantic-deltas.md | 50 ++++++++++++++++++++++++++++++++++++ 5 files changed, 100 insertions(+), 6 deletions(-) diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index 2ffb6bf..3a929a2 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -33,6 +33,16 @@ jobs: - name: Rodar todos os testes run: cargo test --all-features + # O espelho de wasm_rt_memory_t / wasm_rt_funcref_table_t (feature wasm) + # só é conferido contra o wasm-rt.h do wabt quando WASM_RT_INCLUDE existe. + - name: Conferir o layout de wasm-rt.h (wabt 1.0.41) + run: | + mkdir -p target/wabt-include + curl -fsSL https://raw.githubusercontent.com/WebAssembly/wabt/1.0.41/wasm2c/wasm-rt.h \ + -o target/wabt-include/wasm-rt.h + WASM_RT_INCLUDE="$PWD/target/wabt-include" \ + cargo test --features wasm --lib layout_matches_wasm_rt_h + - name: Testar script de proteção de branches run: scripts/test-branch-protection.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index c93ca31..f0e126c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,11 @@ the C ABI or the JSON report must update them. ### Added +- Feature `wasm`: the runtime of Map2Check's `--wasm` mode (D2) ported from the + C runtime: the `wasm_rt_*` stubs, `w2c_wasi__snapshot__preview1_proc_exit` and + `map2check_wasm_malloc`, `map2check_wasm_free`, `map2check_wasm_check_access` + (SD-19). `tests/core_api_coverage.rs` checks every `core_symbol` of + `docs/cutover/abi-mapping.tsv` against the staticlib. - `docs/api-deltas.md` symbol/signature matrix for the core C API and the Map2Check passes, and `docs/semantic-deltas.md` with reproducible behaviour differences, both checked by `tests/compat_docs.rs` (#5). diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 9e3f0d4..119182c 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -128,6 +128,9 @@ build with `--features libfuzzer`. | `map2check_fuzzer_assume` | `void map2check_fuzzer_assume(int32_t condition)` | `map2check_fuzzer_assume(int)` (`nondet_gen_libfuzzer.h:41`) | changed | Feature `libfuzzer`. Same behaviour as `__VERIFIER_assume` (SD-07). | | `__VERIFIER_error` | `void __VERIFIER_error(void)` | — | new | Features `libfuzzer` and `klee`. With an analysis mode set it does nothing, as develop's; with the mode unset it records REACHABILITY, prints the JSON report and aborts (SD-08). | | `__VERIFIER_assert` | `void __VERIFIER_assert(int32_t condition)` | — | new | Features `libfuzzer` and `klee`. `__VERIFIER_assert(0)` calls `__VERIFIER_error()`: nothing with a mode set, an abort without (SD-08). | +| `map2check_wasm_malloc` | `void map2check_wasm_malloc(uint64_t offset, uint64_t size)` | `map2check_wasm_malloc(uint64_t, uint64_t)` (`AnalysisModeMemtrack.c:385`) | same | Feature `wasm` (the header declares it without). Registers the block the lifted `malloc` returned at linear-memory `offset`; 0 and `UINT64_MAX` are ignored, a known offset is resized and revived. Records in every mode (SD-19). | +| `map2check_wasm_free` | `void map2check_wasm_free(uint64_t offset, int64_t line)` | `map2check_wasm_free(uint64_t, int)` (`AnalysisModeMemtrack.c:399`) | same | Feature `wasm`. `line` is `int64_t`, the type the pass declares (`MemoryTrackPass.cpp:829`); develop's C reads the low 32 bits. An offset never registered is `MEMSAFETY_FREE` at `line`; a double free is not an error. Reported where the memory checks are active (SD-19). | +| `map2check_wasm_check_access` | `void map2check_wasm_check_access(uint64_t offset, uint64_t access_size)` | `map2check_wasm_check_access(uint64_t, uint64_t)` (`AnalysisModeMemtrack.c:416`) | same | Feature `wasm`. An access outside every live block is reported as `OVERFLOW`, line 0, no function, exactly as develop does (SD-19). Reported where the memory checks are active. | | `next_current_step` | — | `next_current_step()` | discontinued | Use `get_next_step()`; same effect (#13). | | `map2check_save_memtrack_log` | — | `memtracklog.h:44` | discontinued | Covered by the `map2check_map_*` exports; no external caller. | | `map2check_gen_data_fuzzer_*` | — | bool, char, uchar, short, ushort, int32_t, int64_t, uint, unsigned, long, ulong, float, double, size_t | discontinued | Replaced by `__VERIFIER_nondet_*` (#12). | @@ -167,9 +170,9 @@ Map2Check. | `map2check_nondet_*` | `(u32 line, u32 scope, i32 value, ptr fn)`, `double` value for `double` | pending | Core equivalents: `map2check_save_nondet_log_(line, scope, type_var, T *value, fn)` for every type; the pass stores the value in an `alloca` and passes its address (Task 3.2). See “Nondet generators and the nondet log”. | | `map2check_nondet_cast_*` | — | pending | #21. | | `map2check_pointer_cast` | — | pending | #21. | -| `map2check_wasm_malloc` | — | out-of-scope | Only used by the `feat-wasm-verification` flow (#21). | -| `map2check_wasm_free` | — | out-of-scope | As above. | -| `map2check_wasm_check_access` | — | out-of-scope | As above. | +| `map2check_wasm_malloc` | `(i64, i64)` | same | Core export of the same name and signature, feature `wasm` (Task 2.8, D2). | +| `map2check_wasm_free` | `(i64, i64)` | same | As above. | +| `map2check_wasm_check_access` | `(i64, i64)` | same | As above. | | `__VERIFIER_nondet_*` | `int`, `uint`, `char`, `long`, `ushort`, `pointer` | same | Feature `libfuzzer`. | | `__VERIFIER_assume` | `(int)` | same | Feature `libfuzzer` (SD-07). | | `__VERIFIER_assert` | `(int)` | same | Features `libfuzzer` and `klee` (SD-08). | @@ -179,6 +182,32 @@ The passes also expect the result channel of the embedded library: the files that the Map2Check frontend parses. This library writes them when a mode is set (#19); see “Result files”. +## Feature `wasm`: the wasm2c runtime + +Map2Check's `--wasm` mode lifts a WebAssembly binary to C with wabt's wasm2c +(1.0.41), instruments it with `-wasm-mode` and links the runtime support +(decision D2, ported to Rust instead of dropped). `--features wasm` builds +that support into `libmap2check.a`: + +- the three `map2check_wasm_*` exports of the core matrix, declared in the + header; +- the `wasm_rt_*` functions of `WasmRuntimeStubs.c`, which wasm-rt.h + declares and the lifted code calls, so they are not in `map2check.h` + (`cbindgen:ignore`): `wasm_rt_allocate_memory`, `wasm_rt_grow_memory`, + `wasm_rt_free_memory`, `wasm_rt_allocate_funcref_table`, + `wasm_rt_free_funcref_table`, `wasm_rt_is_initialized`, `wasm_rt_init`, + `wasm_rt_free`, `wasm_rt_trap` and + `w2c_wasi__snapshot__preview1_proc_exit`. Memory and table are calloc/realloc + based (KLEE-friendly), as in develop. + +`wasm_rt_memory_t` and `wasm_rt_funcref_table_t` are mirrored with +`#[repr(C)]` from wabt 1.0.41. `cargo test --features wasm --lib +layout_matches_wasm_rt_h` compares size and field offsets with the real +`wasm-rt.h` when `WASM_RT_INCLUDE` names its directory (CI sets it); without +the variable the test says it is skipped. Link the lifted code against the +wasm-rt.h of wabt 1.0.41; another wabt version may change the layout. +Differences from develop: SD-19. + ## Analysis modes `map2check_set_mode()` replaces the link-time choice of diff --git a/docs/cutover/abi-mapping.tsv b/docs/cutover/abi-mapping.tsv index 3650649..7e2f5dd 100644 --- a/docs/cutover/abi-mapping.tsv +++ b/docs/cutover/abi-mapping.tsv @@ -26,9 +26,9 @@ map2check_check_deref - - check_load já registra a violação (ffi.rs:894, Load map2check_free map2check_check_free+map2check_map_free - C: (ptr name,ptr var_slot,i64 scope,i64 line,ptr fn) (MemoryTrackPass.cpp:818) → núcleo (ambos, Task 2.5): (name,var_slot,ptr,scope:u32,line:u32,fn), var_slot no lugar em que o passe já o envia e ptr = *var_slot; o núcleo é chaveado no endereço de heap carregado, então o passe carrega *var_slot antes da chamada e a ordem é check_free depois map_free; var_slot só nomeia a variável na linha FREE de list_log.csv (AnalysisModeMemtrack.c:339-347), e check_free guarda essa linha quando falha, como o C, que a acrescenta antes de checar (:346-358); golden FALSE-FREE; o passe não chama nenhum dos dois se o valor carregado for NULL, como o C (AnalysisModeMemtrack.c:342); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:352) map2check_free_resolved_address map2check_check_free_resolved_address - chamada única; registra a liberação quando a checagem passa (como o C, :178-182); check_free não registra — o passe chama map_free depois; vale também para a instrumentação de realloc (MemoryTrackPass.cpp:169-182); SD-06 (#37, corrigido): endereço sem registro reporta MEMSAFETY_FREE, como is_invalid_free (ListLog.c:95-118); SD-13: free de ponteiro nunca armazenado é válido no núcleo e FALSE_FREE no C; C: (ptr,i64 line,ptr fn,i1 nullValid) (MemoryTrackPass.cpp:774) → núcleo: (ptr,line:u32,fn,is_null_valid:i16); i1 passa a i16 (zext); C encerra na primeira violação; núcleo registra e continua; SD-03/#36 (Task 2.1) (AnalysisModeMemtrack.c:170) map2check_posix map2check_map_posix_memalign 2.5 chamado em MemoryTrackPass.cpp:120 para posix_memalign; não é obsoleto; C: (ptr,i64 size) (MemoryTrackPass.cpp:793); o C runtime desreferencia ptr (void**) e registra malloc+store_pointer (AnalysisModeMemtrack.c:327-333); núcleo: (ptr_address,size:i32), lê o bloco de *ptr_address e grava malloc e store_pointer(ptr_address,bloco,1,"pointer",1,"cocoiso") como o C; o passe trunca o tamanho para i32 (coerceToParamType); golden POSIX-MEMALIGN -map2check_wasm_malloc map2check_wasm_malloc 2.8 C: (i64,i64) (MemoryTrackPass.cpp:825) -map2check_wasm_free map2check_wasm_free 2.8 C: (i64,i64) (MemoryTrackPass.cpp:829) -map2check_wasm_check_access map2check_wasm_check_access 2.8 C: (i64,i64) (MemoryTrackPass.cpp:833) +map2check_wasm_malloc map2check_wasm_malloc 2.8 C: (i64,i64) (MemoryTrackPass.cpp:825); núcleo igual (u64 offset,u64 size), feature wasm (Task 2.8): AnalysisModeMemtrack.c:385-397 +map2check_wasm_free map2check_wasm_free 2.8 C: (i64,i64) (MemoryTrackPass.cpp:829); núcleo (u64 offset,i64 line), feature wasm: AnalysisModeMemtrack.c:399-407; offset desconhecido reporta MEMSAFETY_FREE na linha (FALSE_FREE) +map2check_wasm_check_access map2check_wasm_check_access 2.8 C: (i64,i64) (MemoryTrackPass.cpp:833); núcleo igual (u64 offset,u64 access_size), feature wasm: AnalysisModeMemtrack.c:416-423; acesso fora de bloco vivo reporta OVERFLOW, linha 0, como o C (FALSE_OVERFLOW; SD-19); só ativo nos modos que checam memória __map2check_main__ __map2check_main__ - definido pelo programa instrumentado, não pela biblioteca overflowError - - morto: errorLines sempre vazio (OverflowPass.cpp:226); C: (i32 line,ptr fn) (OperationsFunctions.hpp:98) reporta FALSE_OVERFLOW e encerra (AnalysisModeOverflow.c:185) __VERIFIER_assert __VERIFIER_assert - diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index a73e440..fc3fd02 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -635,3 +635,53 @@ int __map2check_main__(void) { return 0; } ``` + +### SD-19 — The wasm2c runtime: a bad access is OVERFLOW, and four small differences + +**Classification:** C behaviour preserved on purpose +**Tests:** `rust/src/ffi_tests.rs::wasm_access_past_the_block_is_reported_as_develop_does`, `rust/src/ffi_tests.rs::wasm_free_of_an_untracked_offset_is_a_false_free_at_the_line`, `rust/src/ffi_tests.rs::wasm_double_free_is_not_reported_as_develop`, `rust/src/ffi_tests.rs::wasm_access_inside_a_freed_inner_block_is_covered_by_the_older_block`, `rust/src/ffi_tests.rs::wasm_checks_are_inactive_in_modes_without_memory_checks`, `rust/src/ffi_tests.rs::wasm_first_violation_is_final`, `rust/src/ffi_tests.rs::rt_funcref_table_allocates_zeroed_elements` + +**Rust (feature `wasm`):** `map2check_wasm_check_access(offset, size)` +reports an access that does not lie inside a live registered block as +`OVERFLOW` with line 0 and an empty function name. `map2check_wasm_free` of an +offset that was never registered reports `MEMSAFETY_FREE` at the given line. + +**C runtime of Map2Check `develop`:** `AnalysisModeMemtrack.c:416-423` +writes `FALSE_OVERFLOW` (`write_property(FALSE_OVERFLOW, 0, "")`, reported as +`OVERFLOW`), not `FALSE-DEREF`, for a bad access; `:399-407` writes +`FALSE_FREE` at the line for an unknown offset. `tests/integration/test_wasm_pipeline.sh` +of Map2Check accepts `FALSE_OVERFLOW` for the heap-overflow program. The +library keeps develop's tag on purpose, and the table semantics with it: the +newest row whose range `[offset, offset + size - access + 1)` holds the +offset decides, a freed row has size 0 (so its range is empty and an older +covering block still answers), a double free is not an error, and an access +with no registered block is invalid. The checks are active in the modes that +check memory (memtrack and unset); tracking records in every mode. + +Differences: + +- **No abort without a mode.** As for every check (SD-03, SD-14), the first + violation is final and the run goes on unless an analysis mode is set, + when it writes the result files and aborts. develop always aborts. +- **`line` type.** `map2check_wasm_free` takes `int64_t line`, as the pass + declares it; develop's C parameter is `int`. +- **Trap.** `wasm_rt_trap` records an `UNKNOWN` verdict and aborts (develop + calls `map2check_error()`, which writes no property, so the frontend reads + UNKNOWN). The function never returns, with or without a mode, as wasm-rt.h + declares. +- **Funcref table element size.** develop's `wasm_rt_allocate_funcref_table` + allocates `elements * sizeof(void*)` bytes for elements of + `sizeof(wasm_rt_funcref_t)` (32 bytes), a heap overflow once the module's + element segments are written. The library allocates the real element size. + `max_size` is left untouched in both. +- **Grow.** `wasm_rt_grow_memory` does not clear bytes when the new size is + not larger than the old one (develop would `memset` a wrapped length). + +**Reproduce (link with `--features wasm`, analysis mode memtrack):** + +```c +map2check_set_mode(AnalysisMode_Memtrack); +map2check_init(); +map2check_wasm_malloc(0x10000, 16); +map2check_wasm_check_access(0x10010, 4); /* OVERFLOW, line 0, aborts */ +``` From 34c0a84a5db0e3cb70a6bcd29d2381e3e000d7cb Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 11:00:41 -0400 Subject: [PATCH 49/63] build: bump version to 1.1.0-rc1 and document the cutover changes Add the 1.1.0-rc1 changelog section with the breaking changes, document the runtime-profile and KLEE bitcode build commands, and ship the runtime-profile library (features libfuzzer,wasm) in the release archive. --- CHANGELOG.md | 52 +++++++++++++++++++++++++++++++++ Cargo.lock | 2 +- Cargo.toml | 2 +- Dockerfile | 2 +- docs/integration-guide.md | 20 +++++++++++-- docs/release.md | 7 ++++- klee/Cargo.lock | 2 +- klee/Cargo.toml | 2 +- scripts/package-release.sh | 9 ++++-- scripts/test-package-release.sh | 5 +++- 10 files changed, 92 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f0e126c..2e408c8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,8 +15,49 @@ the C ABI or the JSON report must update them. ## [Unreleased] +## [1.1.0-rc1] - Unreleased + +Release candidate of the cutover work: the library becomes the runtime of +Map2Check (Tasks 1.2, 2.1-2.5, 2.7 and 2.8 of the cutover plan). Read the +breaking changes first. + +### Breaking changes + +- `map2check_check_free` and `map2check_map_free` take one more argument, the + address of the pointer variable (the pointer slot), so the free is logged + like the C runtime does. Passes built against 1.0.0-rc1 must be rebuilt + (`docs/api-deltas.md`). +- `map2check_set_mode` takes an `int32_t`. Invalid modes are rejected. +- Default verdicts changed: the end-of-program leak check (memcleanup) is no + longer opt-in, and the first violation is final; a later passing check does + not clear it (#36). See `docs/semantic-deltas.md`. +- With an analysis mode set, the library prints no JSON, and an unreadable + fuzzer input ends the run instead of continuing. + ### Added +- Runtime profile `runtime` (`cargo build --profile runtime --features + libfuzzer,wasm`): release optimisations with `panic = "unwind"`, so the FFI + guard can recover panics. This is the build Map2Check links. +- Lifecycle: `map2check_init` and every fuzzer iteration reset the analysis + state (#9); every export is guarded against panics and works before + `map2check_init` (#8). +- Runtime analysis modes: `map2check_set_mode` and `map2check_reach_target` + select assert, overflow, memory or reachability checking at run time. +- Legacy result files consumed by Map2Check (property, log and witness files) + and abort on the first violation (#19, #14). +- Core API completion: signed binary operations checked at the operation's + width with `DivByZero`, `posix_memalign` tracking through the `void **` it + wrote (#13), the full `__VERIFIER_nondet_*` family logged as develop writes + it (#6), AFL++-style fuzzer input reading, and rejection of + `__VERIFIER_assume` inside loops. +- KLEE runtime (#20): `klee/Cargo.toml`, a second manifest built with rustc + 1.72.1 (LLVM 16) into one bitcode module by + `scripts/build-klee-runtime.sh`, with CI. The bitcode is built separately + and is not in the release archive. +- `lib/libmap2check_runtime.a` in the release archive: the `runtime` profile + with features `libfuzzer,wasm`. + - Feature `wasm`: the runtime of Map2Check's `--wasm` mode (D2) ported from the C runtime: the `wasm_rt_*` stubs, `w2c_wasi__snapshot__preview1_proc_exit` and `map2check_wasm_malloc`, `map2check_wasm_free`, `map2check_wasm_check_access` @@ -29,6 +70,17 @@ the C ABI or the JSON report must update them. `scripts/package-release.sh` builds the release archive with the C header, both static libraries and the docs (#5). +### Fixed + +- Memory tracking: addresses are validated by their latest alloc/free event + (#11), an untracked resolved free is reported as invalid (#37), a passing + resolved free records the release (#11, #37), and a VLA is a static slot + sized in elements (#38). +- Option semantics: `memcleanup` and null-is-valid are honoured by the memory + checks, and runtime options are separate from the analysis configuration + (#10). +- The memcleanup verdict matches develop's. + ### Changed - `develop` replaces `feature/rust` as the integration branch: CI runs on diff --git a/Cargo.lock b/Cargo.lock index 4cea788..4e5d832 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -180,7 +180,7 @@ checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad" [[package]] name = "map2check-library" -version = "1.0.0-rc1" +version = "1.1.0-rc1" dependencies = [ "cbindgen", "mockall", diff --git a/Cargo.toml b/Cargo.toml index 97bc315..86f32c6 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,7 +1,7 @@ # SPDX-License-Identifier: GPL-3.0-only [package] name = "map2check-library" -version = "1.0.0-rc1" +version = "1.1.0-rc1" edition = "2021" description = "Rust library to support Map2Check tool: tracks memory, basic blocks, and nondeterministic values, and checks safety properties" license = "GPL-3.0-only" diff --git a/Dockerfile b/Dockerfile index 9cc7773..a49a469 100644 --- a/Dockerfile +++ b/Dockerfile @@ -50,4 +50,4 @@ LABEL maintainer="herberthb12@gmail.com" LABEL org.opencontainers.image.title="map2check-library" LABEL org.opencontainers.image.description="Rust library to support Map2Check program verification tool" LABEL org.opencontainers.image.licenses="GPL-3.0-only" -LABEL org.opencontainers.image.version="1.0.0-rc1" +LABEL org.opencontainers.image.version="1.1.0-rc1" diff --git a/docs/integration-guide.md b/docs/integration-guide.md index ddfd71e..ce3cea1 100644 --- a/docs/integration-guide.md +++ b/docs/integration-guide.md @@ -16,6 +16,7 @@ map2check-library--/ ├── include/map2check.h # C header; MAP2CHECK_VERSION == ├── lib/libmap2check.a # plain C programs (no features) ├── lib/libmap2check_libfuzzer.a # LibFuzzer targets (feature libfuzzer) +├── lib/libmap2check_runtime.a # runtime profile, features libfuzzer,wasm ├── docs/integration-guide.md ├── docs/api-deltas.md ├── docs/semantic-deltas.md @@ -34,6 +35,21 @@ cargo build --release # plain C programs cargo build --release --features libfuzzer # LibFuzzer targets (overwrites the same .a) ``` +The build Map2Check links is the `runtime` profile (release optimisations +with `panic = "unwind"`, so the FFI guard can recover panics): + +```sh +cargo build --profile runtime --features libfuzzer,wasm # target/runtime/libmap2check.a +``` + +The runtime of Map2Check's KLEE (`symex`) generator is one LLVM 16 bitcode +module built from `klee/Cargo.toml` with rustc 1.72.1 and the LLVM 16 tools +(`opt-16`, `llvm-dis-16`, ...). It is not part of the release archive: + +```sh +scripts/build-klee-runtime.sh out/map2check_klee.bc +``` + `build.rs` regenerates `include/map2check.h` on every build. The header is not committed (`.gitignore`), so always take it from the same build as the library. @@ -56,12 +72,12 @@ cc -I include my_program.c lib/libmap2check.a -ldl -lpthread -lm -o my_program `undefined reference to '__map2check_main__'`. - The header and the library must come from the same build. The header carries the version as a string, `MAP2CHECK_VERSION` (for example - `"1.0.0-rc1"`), so you can log it or check it in your build: + `"1.1.0-rc1"`), so you can log it or check it in your build: ```c #include "map2check.h" #ifndef MAP2CHECK_VERSION - #error "map2check.h is older than 1.0.0-rc1" + #error "map2check.h is older than 1.1.0-rc1" #endif ``` diff --git a/docs/release.md b/docs/release.md index 15d1727..d0bed3f 100644 --- a/docs/release.md +++ b/docs/release.md @@ -1,7 +1,7 @@ # Release process A release publishes one archive per target with the generated C header, -two static libraries, the integration guide and the compatibility docs. +three static libraries, the integration guide and the compatibility docs. `scripts/package-release.sh` builds it, and CI runs the same script on every push through `scripts/test-package-release.sh`. @@ -16,9 +16,14 @@ from `Cargo.toml` and `` is the `rustc` host triple (for example | `include/map2check.h` | Generated by `build.rs`; `MAP2CHECK_VERSION` must equal ``. | | `lib/libmap2check.a` | `cargo build --release` (no features), for plain C programs. | | `lib/libmap2check_libfuzzer.a` | `cargo build --release --features libfuzzer`, for LibFuzzer targets; the program must define `int __map2check_main__(void)`. | +| `lib/libmap2check_runtime.a` | `cargo build --profile runtime --features libfuzzer,wasm`: the build Map2Check links (unwinding panics, LibFuzzer and wasm2c runtime). | | `docs/integration-guide.md`, `docs/api-deltas.md`, `docs/semantic-deltas.md` | Repository. | | `CHANGELOG.md`, `LICENSE` | Repository. | +The KLEE runtime bitcode is not in the archive: it needs rustc 1.72.1 and +LLVM 16 tools, so it is built separately with +`scripts/build-klee-runtime.sh ` (see the integration guide). + `dist/SHA256SUMS` sits next to the archive. There are two libraries because the `libfuzzer` build references `__map2check_main__`. Without that symbol in the program, linking the `libfuzzer` library fails with diff --git a/klee/Cargo.lock b/klee/Cargo.lock index f235e1b..a01fc7d 100644 --- a/klee/Cargo.lock +++ b/klee/Cargo.lock @@ -10,7 +10,7 @@ checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" [[package]] name = "map2check-klee-runtime" -version = "1.0.0-rc1" +version = "1.1.0-rc1" dependencies = [ "serde", "serde_json", diff --git a/klee/Cargo.toml b/klee/Cargo.toml index 3634452..c5a4e12 100644 --- a/klee/Cargo.toml +++ b/klee/Cargo.toml @@ -22,7 +22,7 @@ # sed -i 's/^version = 4/version = 3/' klee/Cargo.lock [package] name = "map2check-klee-runtime" -version = "1.0.0-rc1" +version = "1.1.0-rc1" edition = "2021" rust-version = "1.72" build = false diff --git a/scripts/package-release.sh b/scripts/package-release.sh index 72a3e6e..b81f2e3 100755 --- a/scripts/package-release.sh +++ b/scripts/package-release.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # SPDX-License-Identifier: GPL-3.0-only -# Monta o arquivo de release (issue #5): header C, as duas bibliotecas -# estáticas, guia de integração, docs de compatibilidade, CHANGELOG e LICENSE. +# Monta o arquivo de release (issue #5): header C, as três bibliotecas +# estáticas (core, libfuzzer, runtime), guia de integração, docs de compatibilidade, CHANGELOG e LICENSE. # Uso: scripts/package-release.sh [DIR_SAIDA] (padrão: dist/) # CHANGELOG_FILE sobrescreve o CHANGELOG verificado (usado pelo teste). set -euo pipefail @@ -29,6 +29,11 @@ cargo build --release --features libfuzzer cp target/release/libmap2check.a "$STAGE/$NAME/lib/libmap2check_libfuzzer.a" cargo build --release cp target/release/libmap2check.a "$STAGE/$NAME/lib/libmap2check.a" +# The build Map2Check links: runtime profile (panic = unwind), libfuzzer + wasm. +# The KLEE bitcode needs rustc 1.72.1 and LLVM 16 (scripts/build-klee-runtime.sh) +# and is built separately; it is not part of this archive. +cargo build --profile runtime --features libfuzzer,wasm +cp target/runtime/libmap2check.a "$STAGE/$NAME/lib/libmap2check_runtime.a" cp include/map2check.h "$STAGE/$NAME/include/" grep -qxF "#define MAP2CHECK_VERSION \"$VERSION\"" "$STAGE/$NAME/include/map2check.h" \ diff --git a/scripts/test-package-release.sh b/scripts/test-package-release.sh index aaed22e..0bb386c 100755 --- a/scripts/test-package-release.sh +++ b/scripts/test-package-release.sh @@ -29,7 +29,7 @@ expected="$(printf '%s\n' \ "$NAME/" "$NAME/CHANGELOG.md" "$NAME/LICENSE" \ "$NAME/docs/" "$NAME/docs/api-deltas.md" "$NAME/docs/integration-guide.md" "$NAME/docs/semantic-deltas.md" \ "$NAME/include/" "$NAME/include/map2check.h" \ - "$NAME/lib/" "$NAME/lib/libmap2check.a" "$NAME/lib/libmap2check_libfuzzer.a" | LC_ALL=C sort)" + "$NAME/lib/" "$NAME/lib/libmap2check.a" "$NAME/lib/libmap2check_libfuzzer.a" "$NAME/lib/libmap2check_runtime.a" | LC_ALL=C sort)" actual="$(tar -tzf "$TMP/dist/$NAME.tar.gz" | LC_ALL=C sort)" if [ "$actual" = "$expected" ]; then ok "conteúdo exato"; else bad "conteúdo: $(diff <(echo "$expected") <(echo "$actual") || true)"; fi @@ -44,6 +44,9 @@ fuzz_syms="$(nm -g --defined-only "$P/lib/libmap2check_libfuzzer.a" 2>/dev/null) if grep -q ' T map2check_init$' <<<"$core_syms" && ! grep -q ' T LLVMFuzzerTestOneInput$' <<<"$core_syms"; then ok "core sem LLVMFuzzerTestOneInput"; else bad "core: símbolos inesperados"; fi if grep -q ' T LLVMFuzzerTestOneInput$' <<<"$fuzz_syms"; then ok "libfuzzer define LLVMFuzzerTestOneInput"; else bad "libfuzzer sem LLVMFuzzerTestOneInput"; fi +rt_syms="$(nm -g --defined-only "$P/lib/libmap2check_runtime.a" 2>/dev/null)" +if grep -q ' T map2check_wasm_malloc$' <<<"$rt_syms" && grep -q ' T LLVMFuzzerTestOneInput$' <<<"$rt_syms"; then ok "runtime tem libfuzzer e wasm"; else bad "runtime sem libfuzzer/wasm"; fi + # Um programa C comum linka e roda só com o conteúdo do pacote mkdir -p "$TMP/c/include" && cp "$P/include/map2check.h" "$TMP/c/include/" sed 's|"../include/map2check.h"|"map2check.h"|' "$ROOT/test/codeAPI_by_caller_test.c" > "$TMP/c/main.c" From c8b3236ab198270bf9f1dab7da110158b5615b46 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 11:05:24 -0400 Subject: [PATCH 50/63] build: bump cbindgen to drop the unsound atty dependency cbindgen 0.26 pulled clap 3 and atty (RUSTSEC-2021-0145). 0.28 builds with rustc 1.82 and, without its default clap feature, pulls neither. 0.29 emits the two enums in another order, so it is not used. build.rs folds the two-space closing #endif that 0.28 writes back to one space; the generated header is byte-identical to the one from 0.26. --- Cargo.lock | 184 +++++++++++++++++------------------------------------ Cargo.toml | 2 +- build.rs | 6 ++ 3 files changed, 64 insertions(+), 128 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 4e5d832..3b83043 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -8,29 +8,6 @@ version = "1.0.14" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000" -[[package]] -name = "atty" -version = "0.2.14" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d9b39be18770d11421cdb1b9947a45dd3f37e93092cbf377614828a319d5fee8" -dependencies = [ - "hermit-abi", - "libc", - "winapi", -] - -[[package]] -name = "autocfg" -version = "1.5.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" - -[[package]] -name = "bitflags" -version = "1.3.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" - [[package]] name = "bitflags" version = "2.13.1" @@ -39,11 +16,10 @@ checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" [[package]] name = "cbindgen" -version = "0.26.0" +version = "0.28.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "da6bc11b07529f16944307272d5bd9b22530bc7d05751717c9d416586cedab49" +checksum = "eadd868a2ce9ca38de7eeafdcec9c7065ef89b42b32f0839278d55f35c54d1ff" dependencies = [ - "clap", "heck", "indexmap", "log", @@ -51,7 +27,7 @@ dependencies = [ "quote", "serde", "serde_json", - "syn 1.0.109", + "syn", "tempfile", "toml", ] @@ -62,36 +38,18 @@ version = "1.0.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" -[[package]] -name = "clap" -version = "3.2.25" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4ea181bf566f71cb9a5d17a59e1871af638180a18fb0035c92ae62b705207123" -dependencies = [ - "atty", - "bitflags 1.3.2", - "clap_lex", - "indexmap", - "strsim", - "termcolor", - "textwrap", -] - -[[package]] -name = "clap_lex" -version = "0.2.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2850f2f5a82cbf437dd5af4d49848fbdfc27c157c3d010345776f952765261c5" -dependencies = [ - "os_str_bytes", -] - [[package]] name = "downcast" version = "0.11.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "1435fa1053d8b2fbbe9be7e97eca7f33d37b28409959813daefc1446a14247f1" +[[package]] +name = "equivalent" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "00d174d5400e5e8fd687ad1049e2f578285fa914201b1af7e8b112a4546bd826" + [[package]] name = "errno" version = "0.3.14" @@ -125,9 +83,9 @@ checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d" [[package]] name = "hashbrown" -version = "0.12.3" +version = "0.16.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8a9ee70c43aaf417c914396645a0fa852624801b24ebb7ae78fe8272889ac888" +checksum = "841d1cc9bed7f9236f321df977030373f4a4163ae1a7dbfe1a51a2c1a51d9100" [[package]] name = "heck" @@ -135,22 +93,13 @@ version = "0.4.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "95505c38b4572b2d910cecb0281560f54b440a19336cbbcb27bf6ce6adc6f5a8" -[[package]] -name = "hermit-abi" -version = "0.1.19" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "62b467343b94ba476dcb2500d242dadbb39557df889310ac77c5d99100aaac33" -dependencies = [ - "libc", -] - [[package]] name = "indexmap" -version = "1.9.3" +version = "2.11.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bd070e393353796e801d209ad339e89596eb4c8d430d18ede6a1cced8fafbd99" +checksum = "4b0f83760fb341a774ed326568e19f5a863af4a952def8c39f9ab92fd95b88e5" dependencies = [ - "autocfg", + "equivalent", "hashbrown", ] @@ -219,15 +168,9 @@ dependencies = [ "cfg-if", "proc-macro2", "quote", - "syn 2.0.118", + "syn", ] -[[package]] -name = "os_str_bytes" -version = "6.6.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e2355d85b9a3786f481747ced0e0ff2ba35213a1f9bd406ed906554d7af805a1" - [[package]] name = "predicates" version = "3.1.4" @@ -278,7 +221,7 @@ version = "0.38.44" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "fdb5bc1ae2baa591800df16c9ca78619bf65c0488b41b96ccec5d11220d8c154" dependencies = [ - "bitflags 2.13.1", + "bitflags", "errno", "libc", "linux-raw-sys", @@ -312,7 +255,7 @@ checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" dependencies = [ "proc-macro2", "quote", - "syn 2.0.118", + "syn", ] [[package]] @@ -329,20 +272,12 @@ dependencies = [ ] [[package]] -name = "strsim" -version = "0.10.0" +name = "serde_spanned" +version = "0.6.9" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "73473c0e59e6d5812c5dfe2a064a6444949f089e20eec9a2e5506596494e4623" - -[[package]] -name = "syn" -version = "1.0.109" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "72b64191b275b66ffe2469e8af2c1cfe3bafa67b529ead792a6d0160888b4237" +checksum = "bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3" dependencies = [ - "proc-macro2", - "quote", - "unicode-ident", + "serde", ] [[package]] @@ -368,27 +303,12 @@ dependencies = [ "windows-sys 0.52.0", ] -[[package]] -name = "termcolor" -version = "1.4.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "06794f8f6c5c898b3275aebefa6b8a1cb24cd2c6c79397ab15774837a0bc5755" -dependencies = [ - "winapi-util", -] - [[package]] name = "termtree" version = "0.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8f50febec83f5ee1df3015341d8bd429f2d1cc62bcba7ea2076759d315084683" -[[package]] -name = "textwrap" -version = "0.16.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c13547615a44dc9c452a8a534638acdf07120d4b6847c8178705da06306a3057" - [[package]] name = "thiserror" version = "2.0.18" @@ -406,54 +326,55 @@ checksum = "ebc4ee7f67670e9b64d05fa4253e753e016c6c95ff35b89b7941d6b856dec1d5" dependencies = [ "proc-macro2", "quote", - "syn 2.0.118", + "syn", ] [[package]] name = "toml" -version = "0.5.11" +version = "0.8.23" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4f7f0dd8d50a853a531c426359045b1998f04219d88799810762cd4ad314234" +checksum = "dc1beb996b9d83529a9e75c17a1686767d148d70663143c7854d8b4a09ced362" dependencies = [ "serde", + "serde_spanned", + "toml_datetime", + "toml_edit", ] [[package]] -name = "unicode-ident" -version = "1.0.24" +name = "toml_datetime" +version = "0.6.11" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" - -[[package]] -name = "winapi" -version = "0.3.9" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" +checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c" dependencies = [ - "winapi-i686-pc-windows-gnu", - "winapi-x86_64-pc-windows-gnu", + "serde", ] [[package]] -name = "winapi-i686-pc-windows-gnu" -version = "0.4.0" +name = "toml_edit" +version = "0.22.27" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" +checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a" +dependencies = [ + "indexmap", + "serde", + "serde_spanned", + "toml_datetime", + "toml_write", + "winnow", +] [[package]] -name = "winapi-util" -version = "0.1.11" +name = "toml_write" +version = "0.1.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" -dependencies = [ - "windows-sys 0.61.2", -] +checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801" [[package]] -name = "winapi-x86_64-pc-windows-gnu" -version = "0.4.0" +name = "unicode-ident" +version = "1.0.24" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" [[package]] name = "windows-link" @@ -552,6 +473,15 @@ version = "0.52.6" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" +[[package]] +name = "winnow" +version = "0.7.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945" +dependencies = [ + "memchr", +] + [[package]] name = "zmij" version = "1.0.21" diff --git a/Cargo.toml b/Cargo.toml index 86f32c6..ac81453 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -51,4 +51,4 @@ inherits = "release" panic = "unwind" [build-dependencies] -cbindgen = "0.26.0" \ No newline at end of file +cbindgen = { version = "0.28.0", default-features = false } \ No newline at end of file diff --git a/build.rs b/build.rs index 1b4e0c5..753b92b 100644 --- a/build.rs +++ b/build.rs @@ -102,6 +102,12 @@ void map2check_binop_shr_unsigned(uint32_t param1, uint32_t param2, uint32_t lin "struct bbtrack_log_t *map2check_save_basic_block_log", "bbtrack_log_t *map2check_save_basic_block_log", ); + // cbindgen 0.28 writes two spaces after the closing `#endif`; 0.26 wrote + // one. Keep the header byte-identical across the bump. + header = header.replace( + "#endif /* __MAP2CHECK_H_INCLUDED__ */", + "#endif /* __MAP2CHECK_H_INCLUDED__ */", + ); let include_guard_end = "#endif /* __MAP2CHECK_H_INCLUDED__ */"; let Some(guard_index) = header.find(include_guard_end) else { panic!("O header gerado não contém o include guard esperado"); From d664e562a173b1260660099a72aef17b76d3acba Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 14:22:51 -0400 Subject: [PATCH 51/63] build: include the wasm runtime in the KLEE bitcode Map2Check's --wasm mode with the symex generator stopped in KLEE with "failed external call: wasm_rt_is_initialized". The KLEE manifest now has the wasm feature and build-klee-runtime.sh builds with --features klee,wasm. tests/klee_paths.sh runs a wasm2c-lifted module against it and expects OVERFLOW at line 0, the verdict of develop's C runtime (SD-19). --- .github/workflows/CI.yml | 2 +- CHANGELOG.md | 1 + docs/api-deltas.md | 2 +- docs/klee-feasibility.md | 2 ++ klee/Cargo.toml | 1 + scripts/build-klee-runtime.sh | 4 ++-- tests/klee/wasm.c | 37 ++++++++++++++++++++++++++++++ tests/klee/wasm.wat | 18 +++++++++++++++ tests/klee_paths.sh | 43 +++++++++++++++++++++++++++++------ 9 files changed, 99 insertions(+), 11 deletions(-) create mode 100644 tests/klee/wasm.c create mode 100644 tests/klee/wasm.wat diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index 3a929a2..5ce95cd 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -190,7 +190,7 @@ jobs: # O clippy em 1.82 do job acima não compila nondet/klee.rs (feature # `klee` só existe em klee/Cargo.toml); este é o lint dele. - name: Clippy do build KLEE (rustc 1.72.1, feature klee) - run: RUSTUP_TOOLCHAIN=1.72.1 cargo clippy --locked --manifest-path klee/Cargo.toml --features klee -- -D warnings + run: RUSTUP_TOOLCHAIN=1.72.1 cargo clippy --locked --manifest-path klee/Cargo.toml --features klee,wasm -- -D warnings # O script roda os autotestes da troca de intrínsecos antes de usá-la: # equivalência por lli-16 em tests/klee/lower-intrinsics.ll e recusa diff --git a/CHANGELOG.md b/CHANGELOG.md index 2e408c8..899970e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -55,6 +55,7 @@ breaking changes first. 1.72.1 (LLVM 16) into one bitcode module by `scripts/build-klee-runtime.sh`, with CI. The bitcode is built separately and is not in the release archive. +- The KLEE bitcode now includes the `wasm` feature (`--features klee,wasm`): the `wasm_rt_*` stubs and the `map2check_wasm_*` exports, so Map2Check's `--wasm` mode with the `symex` generator no longer stops with "failed external call: wasm_rt_is_initialized". `tests/klee_paths.sh` runs a wasm2c-lifted module (`tests/klee/wasm.c`) and expects OVERFLOW at line 0, as develop's C runtime gives (SD-19). - `lib/libmap2check_runtime.a` in the release archive: the `runtime` profile with features `libfuzzer,wasm`. diff --git a/docs/api-deltas.md b/docs/api-deltas.md index 119182c..62fdfd1 100644 --- a/docs/api-deltas.md +++ b/docs/api-deltas.md @@ -206,7 +206,7 @@ layout_matches_wasm_rt_h` compares size and field offsets with the real `wasm-rt.h` when `WASM_RT_INCLUDE` names its directory (CI sets it); without the variable the test says it is skipped. Link the lifted code against the wasm-rt.h of wabt 1.0.41; another wabt version may change the layout. -Differences from develop: SD-19. +Differences from develop: SD-19. The KLEE bitcode (`klee/Cargo.toml`) is built with `--features klee,wasm`, so the stubs and the three exports are in it too (`tests/klee/wasm.c`). ## Analysis modes diff --git a/docs/klee-feasibility.md b/docs/klee-feasibility.md index f7c3be9..ce0b228 100644 --- a/docs/klee-feasibility.md +++ b/docs/klee-feasibility.md @@ -100,6 +100,8 @@ A semente tem esses 30 bytes concatenados. O mesmo `layout.c` foi ligado a `libm 6. **Escopo dos testes.** Os programas imitam a instrumentação dos passes à mão (`tests/klee/m2c.h`). O pipeline real com os passes do Map2Check é a Task 3.x, e o `tests/klee_paths.sh` no CI do Map2Check é a Task 3.4. Overflow, memcleanup, `pchar` e `double` simbólicos não têm programa próprio. `layout.c` cobre os oito tipos inteiros. 7. **CI.** O job `klee_runtime` roda o clippy 1.72.1 sobre o build `klee` (é o único lint de `nondet/klee.rs`), compila o bitcode em 1.72.1 com `--locked`, com os autotestes da troca de intrínsecos, confere que ele existe e que o `llvm-dis-16` o lê, roda o `cargo-deny` sobre `klee/Cargo.lock` e publica o `.bc` como artefato. O KLEE não roda no CI desta biblioteca, porque a imagem com o KLEE 3.1 é a de desenvolvimento do Map2Check e não está num registro público. `tests/klee_paths.sh` roda localmente e entra no CI do Map2Check na Task 3.4. +8. **Modo `--wasm`.** O bitcode `symex` é compilado com `--features klee,wasm`: leva os stubs `wasm_rt_*` e os `map2check_wasm_*`, como o `WasmRuntimeStubs.bc` do develop. Sem eles o KLEE para em `failed external call: wasm_rt_is_initialized` e devolve UNKNOWN. `tests/klee/wasm.c` (módulo liftado por wasm2c) dá `OVERFLOW` na linha 0, igual ao runtime C do develop (SD-19). + ## Custos aceitos (D1, "Custo do pin do rustc 1.72.1") - O que o build `symex` compila (todo o `rust/src` fora dos testes) fica preso ao rustc 1.72. Os ajustes desta task foram: `LazyLock` → `OnceLock` em `state.rs`; `Option::is_none_or` (1.82) → `map_or(true, ..)` em `memtrack/mod.rs`; e `Mutex::clear_poison` sob `#[cfg(not(feature = "klee"))]`, que continua ativo no build normal. O spike também trocou `unsafe extern` por `extern`. Aqui não precisou, porque o único `unsafe extern` fica em `libfuzzer.rs`, que o build `symex` não compila. O job `klee_runtime` falha se algo posterior a 1.72 entrar. diff --git a/klee/Cargo.toml b/klee/Cargo.toml index c5a4e12..07e4076 100644 --- a/klee/Cargo.toml +++ b/klee/Cargo.toml @@ -43,6 +43,7 @@ thiserror = ">=2.0, <=2.0.20" [features] klee = [] +wasm = [] # panic = "abort": nothing catches a panic under KLEE, and the unwind # machinery is code KLEE would have to execute. Fat LTO and one codegen unit diff --git a/scripts/build-klee-runtime.sh b/scripts/build-klee-runtime.sh index 278b22d..36850a1 100755 --- a/scripts/build-klee-runtime.sh +++ b/scripts/build-klee-runtime.sh @@ -5,7 +5,7 @@ # bitcode module (D1, route A', docs/cutover/decisoes.md): # # rustc 1.72.1 (LLVM 16) on klee/Cargo.toml with its own Cargo.lock, -# feature `klee`, panic = abort, fat LTO, codegen-units = 1, +# features `klee,wasm`, panic = abort, fat LTO, codegen-units = 1, # --emit=llvm-bc, then `opt-16 --strip-debug` (condition 6), then the # vector intrinsics KLEE cannot run lowered to scalar code # (scripts/klee-lower-intrinsics.py), then a read-back with `llvm-dis-16`. @@ -76,7 +76,7 @@ log="$work/cargo.log" # a fresh resolution. The profile (klee/Cargo.toml) sets panic = abort, # lto = fat and codegen-units = 1; the check below proves rustc got them. if ! RUSTUP_TOOLCHAIN="$TOOLCHAIN" CARGO_TARGET_DIR="$TARGET_DIR" \ - cargo rustc --locked --release --features klee --lib -v \ + cargo rustc --locked --release --features klee,wasm --lib -v \ --manifest-path "$ROOT/klee/Cargo.toml" -- --emit=llvm-bc >"$log" 2>&1; then cat "$log" >&2 die "cargo rustc failed" diff --git a/tests/klee/wasm.c b/tests/klee/wasm.c new file mode 100644 index 0000000..f4cdea5 --- /dev/null +++ b/tests/klee/wasm.c @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * The --wasm mode of Map2Check: wasm.wat lifted by wasm2c (tests/klee_paths.sh + * generates t.h/t.c), driven like the wrapper of caller.cpp + * generateWasmWrapperStatic (wasm_rt_init, instantiate, _start, free). The + * runtime must supply the wasm_rt_* stubs (KLEE: "failed external call: + * wasm_rt_is_initialized" without them) and the map2check_wasm_* checks. + * Expected: the write one int past the 40-byte block is OVERFLOW at line 0 + * (SD-19), found by KLEE as an abort, in both runtimes. */ +#include +#include +#include "m2c.h" +#include "t.h" + +void map2check_wasm_malloc(uint64_t offset, uint64_t size); +void map2check_wasm_check_access(uint64_t offset, uint64_t access_size); + +/* The two calls MemoryTrackPass -wasm-mode inserts, as the module's imports. */ +void w2c_env_track(struct w2c_env *env, u64 offset, u64 size) { + (void)env; + map2check_wasm_malloc(offset, size); +} +void w2c_env_check(struct w2c_env *env, u64 offset, u64 size) { + (void)env; + map2check_wasm_check_access(offset, size); +} + +int __map2check_main__(void) { + M2C_START(M2C_MODE_MEMTRACK); + w2c_t instance; + wasm_rt_init(); + wasm2c_t_instantiate(&instance, NULL); + w2c_t_0x5Fstart(&instance); + wasm2c_t_free(&instance); + wasm_rt_free(); + map2check_success(); + return 0; +} diff --git a/tests/klee/wasm.wat b/tests/klee/wasm.wat new file mode 100644 index 0000000..2a5d0d8 --- /dev/null +++ b/tests/klee/wasm.wat @@ -0,0 +1,18 @@ +;; SPDX-License-Identifier: GPL-3.0-only +;; A minimal module for tests/klee_paths.sh: what Map2Check's --wasm mode +;; lifts with wasm2c, with the two calls its MemoryTrackPass inserts written +;; as imports (env.track after the allocator, env.check before every access). +;; _start "allocates" the 40-byte block at offset 16, writes inside it, then +;; writes 4 bytes at offset 56, one past the block's end. +(module + (import "env" "track" (func $track (param i64 i64))) + (import "env" "check" (func $check (param i64 i64))) + (memory 1) + (func (export "_start") + (call $track (i64.const 16) (i64.const 40)) + (call $check (i64.const 16) (i64.const 4)) + (i32.store (i32.const 16) (i32.const 42)) + (call $check (i64.const 52) (i64.const 4)) + (i32.store (i32.const 52) (i32.const 7)) + (call $check (i64.const 56) (i64.const 4)) + (i32.store (i32.const 56) (i32.const 999)))) diff --git a/tests/klee_paths.sh b/tests/klee_paths.sh index 4e7d5e5..2324420 100755 --- a/tests/klee_paths.sh +++ b/tests/klee_paths.sh @@ -60,6 +60,7 @@ fi T="$ROOT/tests/klee" mkdir -p "$ROOT/target" W="$(mktemp -d "$ROOT/target/klee-paths.XXXXXX")" +WABT="${WABT:-/opt/wabt-1.0.41}" failures=0 pass() { echo "PASS: $*"; } fail() { @@ -99,12 +100,12 @@ develop_runtime() { # ContainerRealloc NonDetGeneratorKlee WitnessGeneration) case "$1" in target) files+=(AnalysisModeNone) ;; - memtrack) files+=(AnalysisModeMemtrack AllocationLog ListLog HeapLog) ;; + memtrack) files+=(AnalysisModeMemtrack AllocationLog ListLog HeapLog WasmRuntimeStubs) ;; esac mkdir -p "$W/dev" local bcs=() for f in "${files[@]}"; do - [ -f "$W/dev/$f.bc" ] || clang-16 -c -emit-llvm "$lib/$f.c" -o "$W/dev/$f.bc" 2>/dev/null + [ -f "$W/dev/$f.bc" ] || clang-16 -c -emit-llvm -I"$WABT/include" "$lib/$f.c" -o "$W/dev/$f.bc" 2>/dev/null bcs+=("$W/dev/$f.bc") done llvm-link-16 "${bcs[@]}" -o "$2" @@ -120,7 +121,7 @@ develop_runtime() { # run_klee() { local prog="$1" rt="$2" kind="$3" dir="$W/$1-$2-$3" rtbc cflags=() mode=target mkdir -p "$dir" - case "$prog" in deref | vererr_deref) mode=memtrack ;; esac + case "$prog" in deref | vererr_deref | wasm) mode=memtrack ;; esac if [ "$rt" = develop ]; then cflags=(-DM2C_DEVELOP) rtbc="$W/dev-$mode.bc" @@ -128,8 +129,14 @@ run_klee() { else rtbc="$W/rust-rt.bc" fi - clang-16 -g -emit-llvm -c "${cflags[@]}" -I"$T" "$T/$prog.c" -o "$dir/prog.bc" - llvm-link-16 "$dir/prog.bc" "$rtbc" -o "$dir/linked.bc" + clang-16 -g -emit-llvm -c "${cflags[@]}" -I"$T" -I"$W/wasm" -I"$WABT/include" \ + "$T/$prog.c" -o "$dir/prog.bc" + local progs=("$dir/prog.bc") + if [ "$prog" = wasm ]; then # the lifted module, as caller.cpp compiles it + clang-16 -c -emit-llvm -I"$WABT/include" -I"$W/wasm" "$W/wasm/t.c" -o "$dir/lifted.bc" + progs+=("$dir/lifted.bc") + fi + llvm-link-16 "${progs[@]}" "$rtbc" -o "$dir/linked.bc" local extra=() [ "$kind" = frontend ] && extra=(--exit-on-error-type=Abort) (cd "$dir" && klee "${KLEE_FLAGS[@]}" "${extra[@]}" --output-dir=klee-out linked.bc \ @@ -281,6 +288,28 @@ else fail "vererr_target.c: $PATHS_COMPLETED+$PATHS_PARTIAL paths, property '$(cat "$d/map2check_property" 2>/dev/null)'" fi +# ── wasm.c: the --wasm mode, wasm2c-lifted code against the wasm stubs ──────── +echo "== wasm.c" +mkdir -p "$W/wasm" +"$WABT/bin/wat2wasm" "$T/wasm.wat" -o "$W/wasm/t.wasm" +"$WABT/bin/wasm2c" "$W/wasm/t.wasm" -o "$W/wasm/t.c" +run_klee wasm rust full +count wasm rust +run_klee wasm rust frontend +d="$W/wasm-rust-frontend" +if grep -q 'failed external call' "$d/klee.log"; then + fail "wasm.c: KLEE: $(grep 'failed external call' "$d/klee.log" | head -n 1)" +fi +# SD-19: a bad access is OVERFLOW at line 0, with no function (develop: +# write_property(FALSE_OVERFLOW, 0, "")); the same file from both runtimes. +if [ -n "$(abort_ktest "$d")" ] && + [ "$(head -n 1 "$d/map2check_property" 2>/dev/null)" = OVERFLOW ] && + [ "$(sed -n 2p "$d/map2check_property" 2>/dev/null)" = "Line: 0" ]; then + pass "wasm.c: KLEE aborts on the access past the block; property OVERFLOW at line 0" +else + fail "wasm.c: property '$(tr '\n' ' ' <"$d/map2check_property" 2>/dev/null)', abort: '$(abort_ktest "$d")'" +fi + # ── layout.c: ktest -> seed -> Rust libFuzzer reader (condition 9) ──────────── echo "== layout.c" run_klee layout rust full @@ -336,7 +365,7 @@ fi # ── develop's C runtime, same programs ──────────────────────────────────────── if [ -n "${DEVELOP_SRC:-}" ]; then echo "== develop C runtime ($DEVELOP_SRC)" - for prog in prog reach range deref vererr_deref vererr_target layout; do + for prog in prog reach range deref vererr_deref vererr_target wasm layout; do run_klee "$prog" develop full count "$prog" develop run_klee "$prog" develop frontend @@ -348,7 +377,7 @@ fi table="$( echo "| program | Rust runtime (completed+partial) | develop C runtime (completed+partial) |" echo "|---|---|---|" - for prog in prog reach range json deref vererr_deref vererr_target layout; do + for prog in prog reach range json deref vererr_deref vererr_target wasm layout; do echo "| $prog.c | ${COUNT[$prog/rust]:-?} | ${COUNT[$prog/develop]:-not run} |" done )" From 6b49b82ec30931b8a2911d5168e388ac8351d49c Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 15:10:28 -0400 Subject: [PATCH 52/63] fix: write the legacy result files atomically through a temp file and rename Under KLEE the forked states share the real file system and std::fs::write truncates and writes in separate external calls, so one state could empty map2check_property after another wrote its violation. Every legacy file is now written to a temporary file of the same directory and renamed over the target. Names, contents and write order are unchanged. Adds a reader-race test, a rename test and a multi-state KLEE case. --- rust/src/output/legacy_files.rs | 143 ++++++++++++++++++++++++++++++-- tests/klee/multistate.c | 24 ++++++ tests/klee_paths.sh | 23 ++++- 3 files changed, 179 insertions(+), 11 deletions(-) create mode 100644 tests/klee/multistate.c diff --git a/rust/src/output/legacy_files.rs b/rust/src/output/legacy_files.rs index 153f3f7..afe3a45 100644 --- a/rust/src/output/legacy_files.rs +++ b/rust/src/output/legacy_files.rs @@ -124,18 +124,62 @@ fn property_text(result: &AnalysisResult) -> (Option, Option<&'static st } } +/// Writes `contents` to `dir/name` atomically: the full text goes to a +/// temporary file of the same directory, which is then renamed over the +/// target. `std::fs::write` truncates the target and writes in separate +/// calls; under KLEE (`--external-calls=all`) the forked states share the +/// real file system and one state can truncate the file after another wrote +/// its violation, leaving it empty when KLEE halts on the abort. A reader of +/// the target sees the old file or the new one, never an empty or partial +/// one. +/// +/// The temporary name has the pid and a counter. KLEE states +/// share the pid and each has a copy of the counter, so the file is created +/// with `create_new` and a name that already exists is skipped. +pub fn write_atomic(dir: &Path, name: &str, contents: &str) -> io::Result<()> { + use std::io::Write as _; + use std::sync::atomic::{AtomicU64, Ordering}; + static COUNTER: AtomicU64 = AtomicU64::new(0); + + let pid = std::process::id(); + let mut attempts = 0u32; + let (tmp, mut file) = loop { + let n = COUNTER.fetch_add(1, Ordering::Relaxed); + let tmp = dir.join(format!(".{name}.{pid}.{n}.tmp")); + match std::fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&tmp) + { + Ok(file) => break (tmp, file), + Err(e) if e.kind() == io::ErrorKind::AlreadyExists && attempts < 10_000 => { + attempts += 1; + } + Err(e) => return Err(e), + } + }; + let result = file.write_all(contents.as_bytes()).and_then(|()| { + drop(file); + std::fs::rename(&tmp, dir.join(name)) + }); + if result.is_err() { + let _ = std::fs::remove_file(&tmp); + } + result +} + /// Writes the property files of `result` into `dir`, as `write_property` /// and the `write_property_*` helpers of `PropertyGenerator.c` do. pub fn write_property_files(dir: &Path, result: &AnalysisResult) -> io::Result<()> { let (text, extra) = property_text(result); match text { Some(text) => { - std::fs::write(dir.join(PROPERTY_FILE), &text)?; + write_atomic(dir, PROPERTY_FILE, &text)?; if let Some(extra) = extra { - std::fs::write(dir.join(extra), &text)?; + write_atomic(dir, extra, &text)?; } } - None => std::fs::write(dir.join(PROPERTY_FILE_UNKNOWN), "UNKNOWN\n")?, + None => write_atomic(dir, PROPERTY_FILE_UNKNOWN, "UNKNOWN\n")?, } Ok(()) } @@ -154,7 +198,7 @@ pub fn remove_stale_property_files(dir: &Path) { /// `generate_confirmation_file` (`WitnessGeneration.c:28-32`). pub fn write_checked_error(dir: &Path) -> io::Result<()> { - std::fs::write(dir.join(CHECKED_ERROR_FILE), "END OF TIME\n") + write_atomic(dir, CHECKED_ERROR_FILE, "END OF TIME\n") } /// `trackbb_log_to_file` (`TrackBBLog.c:35-44`): `%u;%s\n` per row. @@ -166,7 +210,7 @@ pub fn write_trackbb_log(dir: &Path, rows: &[BasicBlockEntry]) -> io::Result<()> for row in rows.iter().filter(|r| seen.insert(r.line)) { let _ = writeln!(out, "{};{}", row.line, row.function_name); } - std::fs::write(dir.join(TRACKBB_LOG_FILE), out) + write_atomic(dir, TRACKBB_LOG_FILE, &out) } /// The value column and the `enum NONDET_TYPE` number @@ -567,7 +611,7 @@ impl LegacyLogs { r.step ); } - std::fs::write(dir.join(LIST_LOG_FILE), out) + write_atomic(dir, LIST_LOG_FILE, &out) } /// `allocation_log_to_file` (`AllocationLog.c:161-174`). @@ -582,7 +626,7 @@ impl LegacyLogs { i32::from(r.is_free) ); } - std::fs::write(dir.join(ALLOCATION_LOG_FILE), out) + write_atomic(dir, ALLOCATION_LOG_FILE, &out) } /// `heap_log_to_file` (`HeapLog.c:54-66`). @@ -591,7 +635,7 @@ impl LegacyLogs { for r in &self.heap { let _ = writeln!(out, "{};{};{}", c_pointer(r.addr), r.name, r.size); } - std::fs::write(dir.join(HEAP_LOG_FILE), out) + write_atomic(dir, HEAP_LOG_FILE, &out) } /// `nondet_log_to_file` (`NonDetLog.c:17-60`): @@ -606,7 +650,7 @@ impl LegacyLogs { r.line, r.scope, r.function, r.step ); } - std::fs::write(dir.join(NONDET_LOG_FILE), out) + write_atomic(dir, NONDET_LOG_FILE, &out) } } @@ -701,6 +745,87 @@ mod tests { } } + fn bb_rows(count: u32, function: &str) -> Vec { + (1..=count) + .map(|line| BasicBlockEntry::new(u64::from(line), line, function)) + .collect() + } + + /// Race test (the list-ext defect): a reader that polls the target while + /// the writer rewrites it must see the old file or the new one in full, + /// never an empty or partial file. The files are large, so the window + /// between truncating and writing, which `std::fs::write` leaves open, is + /// wide; with the old code this fails on every run seen. + #[test] + fn concurrent_reader_never_sees_empty_or_partial_file() { + use std::sync::atomic::{AtomicBool, Ordering}; + let dir = tempfile::tempdir().unwrap(); + let old = bb_rows(400_000, "old"); + let new = bb_rows(500_000, "newer"); + write_trackbb_log(dir.path(), &old).unwrap(); + let target = dir.path().join(TRACKBB_LOG_FILE); + let old_len = std::fs::metadata(&target).unwrap().len(); + let done = AtomicBool::new(false); + let mut seen = Vec::new(); + std::thread::scope(|s| { + s.spawn(|| { + for _ in 0..20 { + write_trackbb_log(dir.path(), &new).unwrap(); + write_trackbb_log(dir.path(), &old).unwrap(); + } + done.store(true, Ordering::SeqCst); + }); + while !done.load(Ordering::SeqCst) { + if let Ok(m) = std::fs::read(&target) { + seen.push(m.len() as u64); + } + } + }); + let new_len = { + write_trackbb_log(dir.path(), &new).unwrap(); + std::fs::metadata(&target).unwrap().len() + }; + assert!(!seen.is_empty()); + for len in seen { + assert!( + len == old_len || len == new_len, + "a reader saw {len} bytes, expected {old_len} or {new_len}" + ); + } + } + + /// Structural check, deterministic: the target is replaced by a rename + /// (its inode changes), and no temporary file is left behind. + #[test] + fn write_goes_through_a_temp_file_and_rename() { + use std::os::unix::fs::MetadataExt as _; + let dir = tempfile::tempdir().unwrap(); + let target = dir.path().join(PROPERTY_FILE); + write_property_files(dir.path(), &violated(ViolatedProperty::Assert, 5, "main")).unwrap(); + let first = std::fs::metadata(&target).unwrap().ino(); + // An open handle on the old file keeps its inode alive, so a new + // file cannot reuse the number. + let _held = std::fs::File::open(&target).unwrap(); + write_property_files(dir.path(), &violated(ViolatedProperty::Assert, 5, "main")).unwrap(); + assert_ne!(first, std::fs::metadata(&target).unwrap().ino()); + let mut names: Vec = std::fs::read_dir(dir.path()) + .unwrap() + .map(|e| e.unwrap().file_name().into_string().unwrap()) + .collect(); + names.sort(); + assert_eq!(names, ["map2check_property", "map2check_property_assert"]); + } + + #[test] + fn temp_name_collisions_are_skipped() { + let dir = tempfile::tempdir().unwrap(); + for _ in 0..50 { + write_atomic(dir.path(), "f", "x\n").unwrap(); + } + assert_eq!(std::fs::read(dir.path().join("f")).unwrap(), b"x\n"); + assert_eq!(std::fs::read_dir(dir.path()).unwrap().count(), 1); + } + #[test] fn div_by_zero_writes_only_map2check_property() { // PropertyGenerator.c:98-102: no per-property file for FALSE_DIVBYZERO. diff --git a/tests/klee/multistate.c b/tests/klee/multistate.c new file mode 100644 index 0000000..7a3b888 --- /dev/null +++ b/tests/klee/multistate.c @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * Several KLEE states reach the exit path (map2check_success writes the + * property file) and one of them hits a violation. KLEE runs the library's + * file writes as external calls on the real file system, shared by the + * states, so a write that truncates and then writes in separate calls lets + * a state erase the violation another state wrote before KLEE halts on the + * abort (list-ext with --nondet-generator symex: an empty + * map2check_property, reported as UNKNOWN). The files are written through a + * temporary file and a rename, so the property file must end non-empty with + * the violation, and no temporary file may remain. */ +#include "m2c.h" + +int __map2check_main__(void) { + M2C_START(M2C_MODE_TARGET); + int a = NONDET(int, int, 15); + int b = NONDET(int, int, 16); + int c = NONDET(int, int, 17); + int d = NONDET(int, int, 18); + if (a == 1 && b == 2 && c == 3 && d == 4) { + REACH_ERROR(21); + } + map2check_success(); + return 0; +} diff --git a/tests/klee_paths.sh b/tests/klee_paths.sh index 2324420..2e722c1 100755 --- a/tests/klee_paths.sh +++ b/tests/klee_paths.sh @@ -211,6 +211,25 @@ else fail "reach.c: klee_log.csv does not log 42: '$(cat "$d/klee_log.csv" 2>/dev/null)'" fi +# ── multistate.c: many states on the exit path, one violation ──────────────── +# The legacy files are written through a temporary file and a rename, so a +# state that exits cleanly cannot truncate the violation of another one. +echo "== multistate.c" +run_klee multistate rust full +count multistate rust +for rep in 1 2 3; do + run_klee multistate rust frontend + d="$W/multistate-rust-frontend" + if [ -n "$(abort_ktest "$d")" ] && + [ "$(head -n 1 "$d/map2check_property" 2>/dev/null)" = TARGET-REACHED ] && + [ -z "$(find "$d" -maxdepth 1 -name '.*.tmp')" ]; then + pass "multistate.c ($rep): the property file ends with TARGET-REACHED, no temporary file left" + else + fail "multistate.c ($rep): property '$(cat "$d/map2check_property" 2>/dev/null)', tmp: '$(find "$d" -maxdepth 1 -name '.*.tmp')'" + fi + rm -rf "$d" +done + # ── range.c, json.c: formatting a non-unique value (condition 7) ───────────── echo "== range.c" run_klee range rust full @@ -365,7 +384,7 @@ fi # ── develop's C runtime, same programs ──────────────────────────────────────── if [ -n "${DEVELOP_SRC:-}" ]; then echo "== develop C runtime ($DEVELOP_SRC)" - for prog in prog reach range deref vererr_deref vererr_target wasm layout; do + for prog in prog reach multistate range deref vererr_deref vererr_target wasm layout; do run_klee "$prog" develop full count "$prog" develop run_klee "$prog" develop frontend @@ -377,7 +396,7 @@ fi table="$( echo "| program | Rust runtime (completed+partial) | develop C runtime (completed+partial) |" echo "|---|---|---|" - for prog in prog reach range json deref vererr_deref vererr_target wasm layout; do + for prog in prog reach multistate range json deref vererr_deref vererr_target wasm layout; do echo "| $prog.c | ${COUNT[$prog/rust]:-?} | ${COUNT[$prog/develop]:-not run} |" done )" From c18ff116b4bb9c76a53dd984a147e569a44d1fdf Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 16:14:01 -0400 Subject: [PATCH 53/63] build: drop the repo-wide coverage instrumentation from every build .cargo/config.toml added -C instrument-coverage to every cargo build in the repository, including the shipped runtime profile and the KLEE bitcode. Under KLEE the counter update in each basic block cost about a third of the runtime's instructions on list-ext. cargo tarpaulin --engine llvm adds the flag itself, so the coverage gate does not need the file. build-klee-runtime.sh now fails when the crate is compiled with instrument-coverage (from RUSTFLAGS or a config file). --- .cargo/config.toml | 3 --- scripts/build-klee-runtime.sh | 6 ++++++ 2 files changed, 6 insertions(+), 3 deletions(-) delete mode 100644 .cargo/config.toml diff --git a/.cargo/config.toml b/.cargo/config.toml deleted file mode 100644 index c4175c3..0000000 --- a/.cargo/config.toml +++ /dev/null @@ -1,3 +0,0 @@ -[build] -# Enable source-based coverage instrumentation for tarpaulin / llvm-cov -rustflags = ["-C", "instrument-coverage"] diff --git a/scripts/build-klee-runtime.sh b/scripts/build-klee-runtime.sh index 36850a1..1993672 100755 --- a/scripts/build-klee-runtime.sh +++ b/scripts/build-klee-runtime.sh @@ -93,6 +93,12 @@ done case "$crate_line" in *'-C lto=thin'* | *'-C lto=off'* | *'-C lto=no'*) die "LTO is not fat: $crate_line" ;; esac +# Coverage counters (a RUSTFLAGS or .cargo/config.toml `instrument-coverage`) +# put a load/add/store in every basic block that KLEE then executes: about a +# third of the runtime's instructions on list-ext (Task 2.7b). +case "$crate_line" in + *instrument-coverage*) die "the runtime must not be built with coverage instrumentation: $crate_line" ;; +esac mapfile -t modules < <(find "$deps" -maxdepth 1 -name 'map2check-*.bc') [ "${#modules[@]}" -eq 1 ] || die "expected one map2check-*.bc in $deps, found ${#modules[@]}" From 74b63941de2a64d2474eb40a2d33ddf19e4ce4cc Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 16:14:01 -0400 Subject: [PATCH 54/63] perf: check loads without building a hash set per call is_invalid_address built a SipHash set of block addresses on every checked load. Under KLEE that was about 95% of the runtime's instructions on list-ext. It now scans newest first and, for a covering heap block, checks that no newer heap event has the same address. The result is the same: a randomized test compares it with the set-based version. --- rust/src/memtrack/mod.rs | 100 ++++++++++++++++++++++++++++++++++++--- 1 file changed, 93 insertions(+), 7 deletions(-) diff --git a/rust/src/memtrack/mod.rs b/rust/src/memtrack/mod.rs index ea5ec01..a5b29c9 100644 --- a/rust/src/memtrack/mod.rs +++ b/rust/src/memtrack/mod.rs @@ -490,14 +490,32 @@ pub fn is_invalid_address(entries: &[MemTrackEntry], address: usize, size: usize .checked_add(entry.size_destiny) .is_some_and(|allocation_end| address_end <= allocation_end) }; - let mut decided = FixedHashSet::default(); - !entries.iter().rev().any(|entry| { - if entry.is_dynamic || entry.is_free { - decided.insert(entry.points_to) && !entry.is_free && covers(entry) - } else { - entry.size_destiny > 0 && covers(entry) + let is_heap_event = |entry: &MemTrackEntry| entry.is_dynamic || entry.is_free; + // No hash set and no allocation: this runs on every load, and under KLEE + // a set built per call cost about 95% of the runtime's instructions + // (Task 2.7b). An older event of a block is skipped when that block's + // latest event was already looked up; remembering the last such block + // is enough for the common case (one block freed and reallocated in a + // loop) and only saves work, so it cannot change the result. + let mut last_looked_up: Option = None; + for (i, entry) in entries.iter().enumerate().rev() { + if is_heap_event(entry) { + // A freed block (size 0) never validates the access. + if entry.is_free || !covers(entry) || last_looked_up == Some(entry.points_to) { + continue; + } + last_looked_up = Some(entry.points_to); + let is_latest = !entries[i + 1..] + .iter() + .any(|newer| is_heap_event(newer) && newer.points_to == entry.points_to); + if is_latest { + return false; + } + } else if entry.size_destiny > 0 && covers(entry) { + return false; } - }) + } + true } /// Search for the most recent entry with the given `address` as `var_addr`. @@ -761,6 +779,74 @@ mod tests { assert!(is_invalid_address(&live, 0x208, 4)); } + /// The set-based `is_invalid_address` before Task 2.7b, kept as the + /// reference the rewrite must agree with. + fn is_invalid_address_reference( + entries: &[MemTrackEntry], + address: usize, + size: usize, + ) -> bool { + let Some(address_end) = address.checked_add(size) else { + return true; + }; + let covers = |entry: &MemTrackEntry| { + address >= entry.points_to + && entry + .points_to + .checked_add(entry.size_destiny) + .is_some_and(|allocation_end| address_end <= allocation_end) + }; + let mut decided = FixedHashSet::default(); + !entries.iter().rev().any(|entry| { + if entry.is_dynamic || entry.is_free { + decided.insert(entry.points_to) && !entry.is_free && covers(entry) + } else { + entry.size_destiny > 0 && covers(entry) + } + }) + } + + #[test] + fn invalid_address_matches_the_set_based_reference() { + // Random histories over a few block addresses, so blocks are freed, + // reallocated with other sizes and overlap: every kind of entry the + // exports push (malloc, free, stack slot, pointer store). + let mut seed: u64 = 0x2545_F491_4F6C_DD1D; + let mut next = |bound: u64| { + seed ^= seed << 13; + seed ^= seed >> 7; + seed ^= seed << 17; + (seed % bound) as usize + }; + for _ in 0..3000 { + let mut entries = Vec::new(); + for _ in 0..next(24) { + let addr = 0x100 + 8 * next(6); + let size = 4 * next(5); + let entry = match next(4) { + 0 => make_entry(addr, addr, true, false, size), + 1 => { + let mut e = make_entry(addr, addr, true, false, size); + e.set_free(); + e + } + 2 => make_entry(addr, addr, false, false, size), + _ => make_entry(0x900 + addr, addr, false, false, 0), + }; + entries.push(entry); + } + for _ in 0..8 { + let address = 0xF8 + next(64); + let size = next(10); + assert_eq!( + is_invalid_address(&entries, address, size), + is_invalid_address_reference(&entries, address, size), + "address {address:#x} size {size} entries {entries:?}" + ); + } + } + } + #[test] fn invalid_address_keeps_stack_slots_valid() { let entries = vec![make_entry(0x300, 0x300, false, false, 8)]; From 8073d56b29969bf8cfb55a953eff618c04fc0172 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 16:14:01 -0400 Subject: [PATCH 55/63] perf: borrow the function name in VccContext Every checked load and store built a VccContext that copied the function name into an Arc. No checker reads the context, so the copy was pure cost (a malloc, a memcpy and a free per call under KLEE). The context now borrows the name. CHANGELOG records the KLEE throughput work. --- CHANGELOG.md | 7 +++++++ rust/src/analysismode/assert.rs | 2 +- rust/src/analysismode/memory.rs | 2 +- rust/src/analysismode/mod.rs | 18 ++++++++++-------- rust/src/analysismode/overflow.rs | 2 +- 5 files changed, 20 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 899970e..1e41dcc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -81,6 +81,13 @@ breaking changes first. checks, and runtime options are separate from the analysis configuration (#10). - The memcleanup verdict matches develop's. +- Throughput of the KLEE (`symex`) runtime: the load check no longer builds a + hash set per call, the check context borrows the function name, and no + build is instrumented for coverage any more (the repo-wide + `.cargo/config.toml` that added `-C instrument-coverage` is removed; + `cargo tarpaulin` adds it itself, and `scripts/build-klee-runtime.sh` + rejects it). On list-ext the runtime spends about 4 times fewer KLEE + instructions per path and finds the FALSE-DEREF that it missed before. ### Changed diff --git a/rust/src/analysismode/assert.rs b/rust/src/analysismode/assert.rs index 9fd6d33..3f04c08 100644 --- a/rust/src/analysismode/assert.rs +++ b/rust/src/analysismode/assert.rs @@ -32,7 +32,7 @@ mod tests { use super::*; use crate::analysismode::VccContext; - fn ctx() -> VccContext { + fn ctx() -> VccContext<'static> { VccContext::new(12, 0, "foo") } diff --git a/rust/src/analysismode/memory.rs b/rust/src/analysismode/memory.rs index 1f31237..8b82adb 100644 --- a/rust/src/analysismode/memory.rs +++ b/rust/src/analysismode/memory.rs @@ -155,7 +155,7 @@ mod tests { use crate::memtrack::MemTrackEntry; - fn ctx() -> VccContext { + fn ctx() -> VccContext<'static> { VccContext::new(1, 0, "test") } diff --git a/rust/src/analysismode/mod.rs b/rust/src/analysismode/mod.rs index dc93805..3a01300 100644 --- a/rust/src/analysismode/mod.rs +++ b/rust/src/analysismode/mod.rs @@ -1,8 +1,6 @@ // SPDX-License-Identifier: GPL-3.0-only #![deny(unsafe_code)] -use std::sync::Arc; - use crate::error::Map2CheckError; pub mod assert; @@ -11,19 +9,23 @@ pub mod mode; pub mod overflow; /// Context passed to every VCC checker — carries the source location. -#[derive(Debug, Clone)] -pub struct VccContext { +/// +/// The function name is borrowed: a context is built on every checked load +/// and store, and copying the name into the heap there was a measurable cost +/// under KLEE (Task 2.7b). +#[derive(Debug, Clone, Copy)] +pub struct VccContext<'a> { pub line: u32, pub scope: u32, - pub function_name: Arc, + pub function_name: &'a str, } -impl VccContext { - pub fn new(line: u32, scope: u32, function_name: &str) -> Self { +impl<'a> VccContext<'a> { + pub fn new(line: u32, scope: u32, function_name: &'a str) -> Self { Self { line, scope, - function_name: Arc::from(function_name), + function_name, } } } diff --git a/rust/src/analysismode/overflow.rs b/rust/src/analysismode/overflow.rs index fc00eed..6d71cfc 100644 --- a/rust/src/analysismode/overflow.rs +++ b/rust/src/analysismode/overflow.rs @@ -259,7 +259,7 @@ mod tests { use super::*; use crate::analysismode::VccContext; - fn ctx() -> VccContext { + fn ctx() -> VccContext<'static> { VccContext::new(1, 0, "test") } From 843402dd92e26ed6e7086bc842c1916427317e88 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 16:55:48 -0400 Subject: [PATCH 56/63] fix: report a NULL malloc block as UNKNOWN before looking at its size develop's map2check_malloc tests the block first (AnalysisModeMemtrack.c:305-311), so a failed malloc ends the run as UNKNOWN whatever its size. The Rust runtime dropped calls with a negative size first (SD-09). The pass truncates the size to int, so malloc(4 * n) over 2 GiB, which KLEE answers with NULL, reaches the runtime as NULL with a negative size: that state was dropped and ran on, and KLEE bound the next malloc of the same symbolic size to a concrete 2 GiB object. On sv-benchmarks lis_unsafe (symex) this took KLEE to 4.4 GB RSS, against 170 MB with develop's runtime. map2check_map_malloc and map2check_map_calloc now test the block first. A non-NULL block with a negative size is still dropped. tests/klee/hugemalloc.c reproduces the lis_unsafe shape in tests/klee_paths.sh. --- docs/semantic-deltas.md | 5 ++++- rust/src/ffi.rs | 19 +++++++++---------- rust/src/ffi_tests.rs | 22 ++++++++++++++++++++++ tests/klee/hugemalloc.c | 27 +++++++++++++++++++++++++++ tests/klee_paths.sh | 24 ++++++++++++++++++++++-- 5 files changed, 84 insertions(+), 13 deletions(-) create mode 100644 tests/klee/hugemalloc.c diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index fc3fd02..7a4ed65 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -322,7 +322,10 @@ __VERIFIER_error(); /* returns; nothing recorded */ **Rust:** the FFI layer never trusts its arguments. Negative sizes in the `map2check_map_*` calls drop the record, and a `calloc` size that -overflows is dropped too. `set_false_result` ignores an unknown property +overflows is dropped too. A NULL block in `map2check_map_malloc` or +`map2check_map_calloc` is tested first, as develop does, so it stays +`UNKNOWN` in the memory modes whatever the size: a `malloc` over +`i32::MAX` bytes fails, and the pass then passes NULL with a negative size. `set_false_result` ignores an unknown property value. NULL or non-UTF-8 strings become `""`. A negative size in `map2check_check_load` is a `MEMSAFETY_DEREF` violation. Panics are caught at the boundary. diff --git a/rust/src/ffi.rs b/rust/src/ffi.rs index 5e3e5f2..1481894 100644 --- a/rust/src/ffi.rs +++ b/rust/src/ffi.rs @@ -845,7 +845,8 @@ fn malloc_returned_null(ptr_address: *const std::os::raw::c_void) -> bool { /// Records a heap allocation of `size` bytes at `ptr_address`. In the /// memtrack and memcleanup modes a NULL `ptr_address` reports `UNKNOWN` -/// instead, as develop does. +/// instead, as develop does, whatever `size` is; only a non-NULL block with +/// a negative size is dropped (SD-09). /// /// # Safety /// `ptr_address` may be null (null malloc result is valid C). @@ -853,10 +854,9 @@ fn malloc_returned_null(ptr_address: *const std::os::raw::c_void) -> bool { pub unsafe extern "C" fn map2check_map_malloc(ptr_address: *const std::os::raw::c_void, size: i32) { ffi_guard( || { - if size < 0 { - return; - } - if malloc_returned_null(ptr_address) { + // The block before the size, as develop: a failed `malloc(n)` + // with `n` over `i32::MAX` arrives as NULL with a negative size. + if malloc_returned_null(ptr_address) || size < 0 { return; } let addr = ptr_address as usize; @@ -889,7 +889,8 @@ pub unsafe extern "C" fn map2check_map_malloc(ptr_address: *const std::os::raw:: /// Records a heap allocation of `quantity * size` bytes at `ptr_address`. /// In the memtrack and memcleanup modes a NULL `ptr_address` reports -/// `UNKNOWN` instead, as develop's `map2check_calloc` does. +/// `UNKNOWN` instead, as develop's `map2check_calloc` does, whatever the +/// sizes are. /// /// # Safety /// `ptr_address` may be null. @@ -901,10 +902,8 @@ pub unsafe extern "C" fn map2check_map_calloc( ) { ffi_guard( || { - if quantity < 0 || size < 0 { - return; - } - if malloc_returned_null(ptr_address) { + // The block first, as in `map2check_map_malloc`. + if malloc_returned_null(ptr_address) || quantity < 0 || size < 0 { return; } let quantity = quantity as usize; diff --git a/rust/src/ffi_tests.rs b/rust/src/ffi_tests.rs index fbb3797..2843e33 100644 --- a/rust/src/ffi_tests.rs +++ b/rust/src/ffi_tests.rs @@ -1912,6 +1912,28 @@ fn malloc_null_reports_unknown_in_memory_modes_only() { assert_ok(); } +#[test] +fn malloc_null_is_unknown_before_the_size_is_looked_at() { + // develop tests the block before anything else (AnalysisModeMemtrack.c:305-311), + // so a NULL block is UNKNOWN whatever the size. The pass truncates the + // size to `int`: `malloc(4 * n)` that fails because it is over 2 GiB + // arrives here as NULL with a negative size. Dropping that call as SD-09 + // let the program run on with the NULL block (task-2.7c-report.md). + let _guard = fresh(); + for mode in [AnalysisMode::Memtrack, AnalysisMode::Memcleanup] { + start(mode); + // SAFETY: a NULL block is only compared. + unsafe { map2check_map_malloc(std::ptr::null(), i32::MIN + 0x0100_0000) }; + assert_violated(ViolatedProperty::Unknown, 0, ""); + for (quantity, size) in [(-1, 4), (2, -4)] { + start(mode); + // SAFETY: as above. + unsafe { map2check_map_calloc(std::ptr::null(), quantity, size) }; + assert_violated(ViolatedProperty::Unknown, 0, ""); + } + } +} + #[test] fn options_default_off() { let _guard = fresh(); diff --git a/tests/klee/hugemalloc.c b/tests/klee/hugemalloc.c new file mode 100644 index 0000000..b93065c --- /dev/null +++ b/tests/klee/hugemalloc.c @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * The shape of sv-benchmarks lis_unsafe: two mallocs of 4 * n bytes with a + * symbolic n. KLEE returns NULL for a size over 2^31; the pass truncates + * the size to int, so the runtime sees NULL with a negative size, and must + * report UNKNOWN (develop's map2check_malloc) instead of letting the state + * run on to the second malloc, which KLEE then binds to a concrete 2 GiB + * object (task-2.7c-report.md). */ +#include +#include "m2c.h" + +int __map2check_main__(void) { + M2C_START(M2C_MODE_MEMTRACK); + int n = NONDET(int, int, 10); + int *best = malloc(sizeof(int) * (long)n); + TRACK_MALLOC(best, (int)(sizeof(int) * (long)n), 11); + int *prev = malloc(sizeof(int) * (long)n); + TRACK_MALLOC(prev, (int)(sizeof(int) * (long)n), 12); + int v = 0; + if (n > 0) { + CHECK_LOAD(&best[0], (int)sizeof(int), 15); + v = best[0]; + } + free(best); + free(prev); + map2check_success(); + return v; +} diff --git a/tests/klee_paths.sh b/tests/klee_paths.sh index 2e722c1..1796864 100755 --- a/tests/klee_paths.sh +++ b/tests/klee_paths.sh @@ -121,7 +121,7 @@ develop_runtime() { # run_klee() { local prog="$1" rt="$2" kind="$3" dir="$W/$1-$2-$3" rtbc cflags=() mode=target mkdir -p "$dir" - case "$prog" in deref | vererr_deref | wasm) mode=memtrack ;; esac + case "$prog" in deref | hugemalloc | vererr_deref | wasm) mode=memtrack ;; esac if [ "$rt" = develop ]; then cflags=(-DM2C_DEVELOP) rtbc="$W/dev-$mode.bc" @@ -283,6 +283,26 @@ else fail "deref.c: the abort's ktest does not have i = 4" fi +# ── hugemalloc.c: a failed malloc of a symbolic size is UNKNOWN ────────────── +# lis_unsafe: KLEE returns NULL for 4 * n > 2^31 and the pass hands the +# runtime a negative int size. The NULL block must end the run as UNKNOWN, +# as develop does; dropping the call let the state reach the second malloc, +# which KLEE bound to a concrete 2 GiB object (4.4 GB RSS, task 2.7c). +echo "== hugemalloc.c" +run_klee hugemalloc rust frontend +d="$W/hugemalloc-rust-frontend" +if [ -n "$(abort_ktest "$d")" ] && [ -e "$d/map2check_property__unknown" ] && + [ ! -e "$d/map2check_property" ]; then + pass "hugemalloc.c: the NULL block of a huge malloc aborts as UNKNOWN" +else + fail "hugemalloc.c: abort '$(abort_ktest "$d")', files: $(ls "$d" | tr '\n' ' ')" +fi +if grep -q 'Large alloc\|over memory cap' "$d/klee.log"; then + fail "hugemalloc.c: KLEE: $(grep 'Large alloc\|over memory cap' "$d/klee.log" | head -n 1)" +else + pass "hugemalloc.c: no state reaches a concrete multi-GiB object" +fi + # ── vererr_*.c: __VERIFIER_error/_assert are no-ops with a mode set ───────── echo "== vererr_deref.c" run_klee vererr_deref rust full @@ -384,7 +404,7 @@ fi # ── develop's C runtime, same programs ──────────────────────────────────────── if [ -n "${DEVELOP_SRC:-}" ]; then echo "== develop C runtime ($DEVELOP_SRC)" - for prog in prog reach multistate range deref vererr_deref vererr_target wasm layout; do + for prog in prog reach multistate range deref hugemalloc vererr_deref vererr_target wasm layout; do run_klee "$prog" develop full count "$prog" develop run_klee "$prog" develop frontend From 44585e885d2b146dd3b4082d05220fbd5fa10ca6 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 17:00:15 -0400 Subject: [PATCH 57/63] test: check the huge-malloc abort by its ktest, not by the files other states leave In the frontend run other KLEE states also write property files (the n == 0 path ends with NONE), so the absence of map2check_property proved nothing. The check now reads n from the abort's ktest and requires 4 * n to be over 2^31, and the program frees its blocks so no state ends as FALSE-MEMTRACK. --- tests/klee/hugemalloc.c | 2 ++ tests/klee_paths.sh | 17 +++++++++++++---- 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/tests/klee/hugemalloc.c b/tests/klee/hugemalloc.c index b93065c..77dae50 100644 --- a/tests/klee/hugemalloc.c +++ b/tests/klee/hugemalloc.c @@ -20,7 +20,9 @@ int __map2check_main__(void) { CHECK_LOAD(&best[0], (int)sizeof(int), 15); v = best[0]; } + TRACK_FREE(best, 18); free(best); + TRACK_FREE(prev, 20); free(prev); map2check_success(); return v; diff --git a/tests/klee_paths.sh b/tests/klee_paths.sh index 1796864..1a2159a 100755 --- a/tests/klee_paths.sh +++ b/tests/klee_paths.sh @@ -291,11 +291,20 @@ fi echo "== hugemalloc.c" run_klee hugemalloc rust frontend d="$W/hugemalloc-rust-frontend" -if [ -n "$(abort_ktest "$d")" ] && [ -e "$d/map2check_property__unknown" ] && - [ ! -e "$d/map2check_property" ]; then - pass "hugemalloc.c: the NULL block of a huge malloc aborts as UNKNOWN" +k="$(abort_ktest "$d")" +n= +if [ -n "$k" ]; then + hex="$(ktest_objects "$k" "$W/seed.tmp" | sed -n 's/^non_det_int 4 \(........\)$/\1/p')" + # little-endian int32 + [ -n "$hex" ] && n=$((0x${hex:6:2}${hex:4:2}${hex:2:2}${hex:0:2})) + [ -n "$n" ] && [ "$n" -ge $((1 << 31)) ] && n=$((n - (1 << 32))) +fi +# The abort is the huge malloc's: 4 * n is negative or over 2^31. +if [ -n "$n" ] && { [ "$n" -lt 0 ] || [ "$n" -gt $((1 << 29)) ]; } && + [ -e "$d/map2check_property__unknown" ]; then + pass "hugemalloc.c: the NULL block of a huge malloc (n = $n) aborts as UNKNOWN" else - fail "hugemalloc.c: abort '$(abort_ktest "$d")', files: $(ls "$d" | tr '\n' ' ')" + fail "hugemalloc.c: abort '$k' with n = '$n', files: $(ls "$d" | tr '\n' ' ')" fi if grep -q 'Large alloc\|over memory cap' "$d/klee.log"; then fail "hugemalloc.c: KLEE: $(grep 'Large alloc\|over memory cap' "$d/klee.log" | head -n 1)" From e52e6db43f2181fe2a28579f8be202e5f03a4e2d Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 22:03:23 -0400 Subject: [PATCH 58/63] docs: record D3 and D4 as approved and set the RSS limit to 1.36 --- docs/cutover/decisoes.md | 24 +++++++++++++++++++ .../2026-10-15-cutover-runtime-c-map2check.md | 4 +++- 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/docs/cutover/decisoes.md b/docs/cutover/decisoes.md index 303872a..759fe4a 100644 --- a/docs/cutover/decisoes.md +++ b/docs/cutover/decisoes.md @@ -152,3 +152,27 @@ A aprovação aceita a proposta junto com estas condições. **Status, lado KLEE: ATENDIDA (Task 2.7, 2026-10-16).** O gerador KLEE do Rust nomeia os objetos `non_det_`, como `develop`, com `sizeof(tipo)` bytes. Em `layout.c` (`char`, `short`, `ushort`, `int`, `uint`, `long`, `ulong`, `uchar`), o ktest do caminho de erro tem os oito objetos em ordem e little-endian. Convertido em semente como `ktestToFuzzerBytes` faz (30 bytes concatenados) e rodado como argumento pelo leitor libFuzzer do Rust (`--features libfuzzer`, perfil `runtime`), ele chega ao mesmo erro (`TARGET-REACHED`). Uma semente de zeros do mesmo tamanho não chega. O `pchar` do KLEE passou a limitar o comprimento a 4096, como o leitor AFL++, para que o replay gere a mesma string. 10. **O rustc 1.72.1 (2023) fica preso ao LLVM 16.** Quando o Map2Check migrar de LLVM, o toolchain do `symex` precisa acompanhar ou ser removido pela rota C. 11. **O programa analisado é compilado com `-fexceptions`.** Responsável: Task 3.2. Um `__VERIFIER_assume` falso dentro de uma iteração do fuzzer desenrola a pilha (`resume_unwind`) através dos frames C do programa até o `LLVMFuzzerTestOneInput`. Sem `-fexceptions`, o clang marca toda função C como `nounwind`, e desenrolar a partir de uma função `nounwind` é comportamento indefinido no IR do LLVM. Hoje funciona só porque o x86-64 Linux emite `.eh_frame` por padrão. Com `-fno-asynchronous-unwind-tables -fno-unwind-tables` o processo aborta (“failed to initiate panic, error 5”), e o AFL++ contaria isso como crash: um falso positivo. O passo que gera o bitcode do programa (`clang -emit-llvm`) tem de passar `-fexceptions`, porque os atributos viajam no bitcode que o `afl-clang-fast` recebe. Nenhuma etapa pode acrescentar `-fno-unwind-tables` ou `-fno-asynchronous-unwind-tables`. Unwind tables sozinhas (`-funwind-tables`) não bastam: fazem o desenrolar funcionar, mas o IR continua `nounwind`. Evidência: `tests/ffi_harness.c` em `-O2` com clang-16 e afl-clang-fast (`scripts/cutover/d1-remeasure/results/flags16.txt`, tabela em `docs/api-deltas.md`, “`__VERIFIER_assume` under a fuzzer”). O CI roda o harness em `-O2` com e sem `-fexceptions`. + +## D3 + +**Pergunta:** a opção `--btree` (armazenamento em disco do runtime C) é portada para o runtime Rust, ou removida com justificativa. + +**Status: APROVADA pelo mantenedor em 2026-10-16** (Task 4.3, gate 4.2 execução 1; `Map2Check-rust/tests/cutover/gate-report.md`) + +**Decisão:** `--btree` **não é portada**. Continua um no-op com aviso (comportamento atual do frontend Rust). + +**Justificativa:** +- O `--btree` do C não é referência válida. Sob `symex`, dá SUCCEEDED errado em 3 tarefas cujo veredito esperado é falso: `lis_unsafe`, `memleaks_test1-1` e `memleaks_test17_2-1`. Também perde `list-ext` (FAILED → UNKNOWN, nos dois geradores). +- Números de pico de RSS (mediana, 30 tarefas `valid-memsafety`): symex C-btree 136,3 MB contra Rust 221,2 MB; afl C-btree 94,3 MB contra Rust 92,3 MB. O maior pico do Rust é 472 MB. + +**Desvio da condição literal da Task 2.6:** a condição (RSS do Rust ≤ o do C com `--btree` em todas as tarefas) **não** é atendida: sob `symex` o Rust fica acima em 30 de 30 tarefas, por um excesso fixo de cerca de 80 MB. O mantenedor aprovou explicitamente se afastar dela, pelo motivo acima. + +## D4 + +**Pergunta:** limite do pico de RSS do runtime Rust em relação ao C. + +**Status: APROVADA pelo mantenedor em 2026-10-16** + +**Decisão:** aceitar a razão mediana medida. O limite do critério 8 passa de 1,25 para **1,36** (medido 1,360; afl 0,99, symex 1,58). `compare.py` usa `--max-rss` 1.36 por padrão. + +**Causa:** o excesso sob `symex` é quase constante (cerca de +80 MB). A hipótese é que o KLEE carrega e otimiza um `map2check-klee.bc` de 2,2 MB, contra 0,13 MB dos `.bc` do runtime C. **Hipótese não perfilada.** diff --git a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md index d64ca6d..295123e 100644 --- a/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md +++ b/docs/superpowers/plans/2026-10-15-cutover-runtime-c-map2check.md @@ -100,7 +100,7 @@ Os critérios são medidos pelo gate da Task 4.2 sobre o corpus `tests/regressio **Desempenho:** 6. A mediana, entre tarefas, da razão `wall_rust / wall_c` é ≤ 1,10. **Bloqueia.** 7. A contagem de TIMEOUT por gerador não aumenta. **Bloqueia.** -8. A mediana da razão de pico de RSS é ≤ 1,25 (D4). **Bloqueia** depois de aprovado; antes disso, só alerta. +8. A mediana da razão de pico de RSS é ≤ 1,36 (D4, aprovada em 2026-10-16; era 1,25 pendente de aprovação). **Bloqueia.** 9. Uma tarefa isolada com razão > 1,5 não bloqueia se os critérios 6 e 7 passarem, mas exige perfil registrado em `docs/cutover/perf.md`. **Qualidade da biblioteca (já existentes; continuam obrigatórios):** `cargo test --all-features`, `cargo clippy -- -D warnings`, `cargo fmt --check`, `cargo deny check`, gate de cobertura e `tests/ffi_harness.c` sob ASan/UBSan sem achados. @@ -937,6 +937,8 @@ fn posix_memalign_is_tracked_like_malloc() { **Critério de conclusão:** D3 registrada com números, e a implementação ou o no-op testado. +**Resultado (2026-10-16):** D3 aprovada. `--btree` não é portada e fica como no-op com aviso. A condição literal do Step 2 não foi atendida (Rust acima do C-btree em 30 de 30 tarefas `symex`), e o mantenedor aprovou o desvio: o `--btree` do C dá SUCCEEDED errado em `lis_unsafe`, `memleaks_test1-1` e `memleaks_test17_2-1`, então não é referência válida (ver `docs/cutover/decisoes.md`). + ### Task 2.7: Gerador KLEE — #20 **Files:** conforme D1. Na rota A′: `klee-rt/` (crate `no_std` separada no workspace) + `docs/klee-feasibility.md`. From 6094f27eecbf56a5187b2515ab71a50dc9f42d45 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 22:21:53 -0400 Subject: [PATCH 59/63] docs: add SD-20 and SD-21 for the gate's C-bug deltas and record D2 and D5 SD-20 (C-BUG-property-race): the C runtime truncates the shared map2check_property from other KLEE states; the Rust library writes through a temp file and rename. SD-21 (C-BUG-oom-afl-nondet): the C NonDetLog grows across AFL++ persistent iterations until the 3 GB cap; the cause is read from the source, not profiled. decisoes.md gains D2 (port wasm, plan recommendation, executed in 2.8 and 2.8b, formal approval pending) and D5 (no compatibility layer, user decision of 2026-10-15), and a measured note under D4. --- docs/cutover/decisoes.md | 37 +++++++++++++ docs/semantic-deltas.md | 110 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 147 insertions(+) diff --git a/docs/cutover/decisoes.md b/docs/cutover/decisoes.md index 759fe4a..f62e24f 100644 --- a/docs/cutover/decisoes.md +++ b/docs/cutover/decisoes.md @@ -153,6 +153,21 @@ A aprovação aceita a proposta junto com estas condições. 10. **O rustc 1.72.1 (2023) fica preso ao LLVM 16.** Quando o Map2Check migrar de LLVM, o toolchain do `symex` precisa acompanhar ou ser removido pela rota C. 11. **O programa analisado é compilado com `-fexceptions`.** Responsável: Task 3.2. Um `__VERIFIER_assume` falso dentro de uma iteração do fuzzer desenrola a pilha (`resume_unwind`) através dos frames C do programa até o `LLVMFuzzerTestOneInput`. Sem `-fexceptions`, o clang marca toda função C como `nounwind`, e desenrolar a partir de uma função `nounwind` é comportamento indefinido no IR do LLVM. Hoje funciona só porque o x86-64 Linux emite `.eh_frame` por padrão. Com `-fno-asynchronous-unwind-tables -fno-unwind-tables` o processo aborta (“failed to initiate panic, error 5”), e o AFL++ contaria isso como crash: um falso positivo. O passo que gera o bitcode do programa (`clang -emit-llvm`) tem de passar `-fexceptions`, porque os atributos viajam no bitcode que o `afl-clang-fast` recebe. Nenhuma etapa pode acrescentar `-fno-unwind-tables` ou `-fno-asynchronous-unwind-tables`. Unwind tables sozinhas (`-funwind-tables`) não bastam: fazem o desenrolar funcionar, mas o IR continua `nounwind`. Evidência: `tests/ffi_harness.c` em `-O2` com clang-16 e afl-clang-fast (`scripts/cutover/d1-remeasure/results/flags16.txt`, tabela em `docs/api-deltas.md`, “`__VERIFIER_assume` under a fuzzer”). O CI roda o harness em `-O2` com e sem `-fexceptions`. +## D2 + +**Pergunta:** o modo `--wasm` entra no cutover? + +**Status: recomendação do plano, executada.** O plano registra a D2 só como recomendação ("Portar"), sem aprovação explícita do mantenedor. As Tasks 2.8 e 2.8b a seguiram. Falta a aprovação formal. + +**Decisão (texto do plano):** portar os stubs do wasm2c para Rust, e não deixar `--wasm` fora do escopo. + +**Justificativa (texto do plano):** o CI do Map2Check tem o job `e2e-wasm` ativo (`ci.yml:213`). Não há base, portanto, para declarar o modo obsoleto. + +**Execução:** +- Task 2.8: `rust/src/wasm.rs` (feature `wasm`), com a semântica do develop registrada no SD-19. +- Task 2.8b: o runtime KLEE (`map2check-klee.bc`) também é construído com a feature `wasm`. +- Task 3.3: o frontend deixa de ligar `WasmRuntimeStubs.bc`. Os testes `test_wasm_*.sh` passam com o runtime Rust, nos geradores `afl` e `symex`. + ## D3 **Pergunta:** a opção `--btree` (armazenamento em disco do runtime C) é portada para o runtime Rust, ou removida com justificativa. @@ -176,3 +191,25 @@ A aprovação aceita a proposta junto com estas condições. **Decisão:** aceitar a razão mediana medida. O limite do critério 8 passa de 1,25 para **1,36** (medido 1,360; afl 0,99, symex 1,58). `compare.py` usa `--max-rss` 1.36 por padrão. **Causa:** o excesso sob `symex` é quase constante (cerca de +80 MB). A hipótese é que o KLEE carrega e otimiza um `map2check-klee.bc` de 2,2 MB, contra 0,13 MB dos `.bc` do runtime C. **Hipótese não perfilada.** + +**Perfil posterior (critério 9, 2026-10-16):** a hipótese foi medida em byte_add_bug, com KLEE rodando sozinho no `-witness-result.bc` mantido. +- **Pico de RSS do KLEE:** C 127 MB, Rust 221 MB. +- **Sem o simbolizador de backtrace da std:** 161 MB. Os corpos de gimli, addr2line, miniz_oxide, rustc_demangle e backtrace_rs foram removidos com `llvm-extract --delete`. +- **Conclusão:** cerca de 60 dos 94 MB de excesso vêm desse código, que nunca roda. +- **Detalhes:** `Map2Check-rust/tests/cutover/gate-report.md`, "Critério 9 — perfis". + +## D5 + +**Pergunta:** que ABI os passes do Map2Check emitem depois do cutover. + +**Status: DECIDIDA pelo usuário em 2026-10-15**, na sessão que pediu o plano. O plano a registra como decidida (seção 2 e "Global Constraints"). + +**Decisão:** sem camada de compatibilidade. +- A biblioteca Rust não exporta os nomes nem as assinaturas do runtime C. Não há feature `compat`, módulo `compat.rs` nem aliases. +- Os passes LLVM migram diretamente para a API núcleo, sem ABI dupla. A #23 entra no cutover (Task 3.2). +- A #21 fecha como "won't do (D5)". + +**Consequências:** +- A diferença de nomes entre o runtime C e o header Rust se resolve no que os passes emitem, com o mapeamento de `docs/cutover/abi-mapping.tsv` (Tasks 0.3 e 3.2), e não na biblioteca. +- A Task 2.5 só acrescenta à API núcleo, com nomes núcleo, o que ainda faltava. Um teste falha se um nome do runtime C aparecer no artefato. +- A antiga Fase 7 (aposentar a camada de compatibilidade) foi removida do plano. diff --git a/docs/semantic-deltas.md b/docs/semantic-deltas.md index 7a4ed65..e36f142 100644 --- a/docs/semantic-deltas.md +++ b/docs/semantic-deltas.md @@ -688,3 +688,113 @@ map2check_init(); map2check_wasm_malloc(0x10000, 16); map2check_wasm_check_access(0x10010, 4); /* OVERFLOW, line 0, aborts */ ``` + +### SD-20 — Result files survive other KLEE states (gate ref `C-BUG-property-race`) + +**Classification:** intentional +**Tests:** `rust/src/output/legacy_files.rs::concurrent_reader_never_sees_empty_or_partial_file`, `rust/src/output/legacy_files.rs::write_goes_through_a_temp_file_and_rename`, `rust/src/output/legacy_files.rs::temp_name_collisions_are_skipped` + +This entry covers a bug of the C runtime of Map2Check `develop` that the +Rust library fixes. The cutover gate cites it as `C-BUG-property-race` in +`tests/cutover/deltas-aceitos.tsv` of Map2Check `feat/rust-runtime` (10 +rows: verdict, tag, line and function of `simple-ext` and `ex01_false` +under `unreach-call`/`symex`, plus line and function of `960521-1_1-1` +under `valid-memsafety`/`symex`). + +**Rust:** every legacy result file (`map2check_property`, its `_deref`, +`_free`, `_memtrack` and `_memcleanup` variants, and the logs) is written +to a temporary file in the same directory and then renamed over the target +(`write_atomic`, `6b49b82`). A reader, or another KLEE state, sees either +the old content or the new content, never an empty file. Names, contents +and write order are the same as develop's. + +**C runtime of Map2Check `develop`:** `write_property` +(`PropertyGenerator.c:50`) does `fopen("map2check_property", "w")`, then +`fprintf`, then `fclose`. Under KLEE with `--external-calls=all`, every +state shares the host file. A state that later opens the file with `"w"` +truncates what the violating state wrote. `--exit-on-error-type=Abort` then +halts KLEE before that state writes its own content. The frontend finds a +0-byte `map2check_property`. + +Effects in gate run 1 (task 4.2), deterministic in 5 of 5 C reps: +- `simple-ext` and `ex01_false`: KLEE reaches `__VERIFIER_error` + (`abort.err` is written, and so is `map2check_checked_error`), but the + frontend reports UNKNOWN. The expected verdict is false; Rust reports + FAILED `TARGET-REACHED`. +- `960521-1_1-1`: `map2check_property` and `map2check_property_deref` are + both 0 bytes. The frontend still says FALSE-DEREF, because the `_deref` + file exists, but it parses line 0 and an empty function. C under `afl` + reports `533 main` for the same task, and so does Rust under `symex`. + +The Rust library had the same race before `6b49b82` (task 3.3, "Cause B"). + +**Reproduce:** the program is any one where KLEE forks a state that +reaches the target and states that do not. With the C runtime: +`map2check --debug --nondet-generator symex --target-function ex01_false.c` +(sv-benchmarks `map2check-assert`). The kept directory has `abort.err` in +`klee-out-0`, a 12-byte `map2check_checked_error`, and a 0-byte +`map2check_property`. Under the Rust runtime, `tests/klee/multistate.c` +(`tests/klee_paths.sh`) checks the opposite. + +```c +/* Under KLEE: one path reaches the target, the other finishes later. */ +int main(void) { + int x = __VERIFIER_nondet_int(); + if (x == 42) + __VERIFIER_error(); /* C: map2check_property may end up 0 bytes */ + return 0; /* Rust: map2check_property keeps TARGET-REACHED */ +} +``` + +### SD-21 — AFL++ iterations do not accumulate nondet records (gate ref `C-BUG-oom-afl-nondet`) + +**Classification:** intentional +**Tests:** `rust/src/nondet/libfuzzer.rs::two_fuzzer_iterations_yield_independent_results`, `rust/src/nondet/libfuzzer.rs::fuzzer_iteration_resets_analysis_state_and_input` + +This entry covers a bug of the C runtime of Map2Check `develop` that the +Rust library does not have. The cutover gate cites it as +`C-BUG-oom-afl-nondet` (3 rows: the verdict of `standard_sentinel-2`, +`standard_two_index_04` and `standard_two_index_05` under +`unreach-call`/`afl`). + +**Rust:** `LLVMFuzzerTestOneInput` resets the analysis state, the nondet +records included, before every iteration (SD-11). Peak RSS on the three +tasks is about 341 MB, the AFL++ process plateau. The verdict is UNKNOWN, +the same as C's uncapped verdict. + +**C runtime of Map2Check `develop`:** each of these programs reads 100 000 +`__VERIFIER_nondet_int` values per run. +- With the gate's 3 GB container cap, C is OOM-killed in 15 of 15 reps + (`oom_kill` in the cgroup's `memory.events`, maxrss about 3.1 GB). The + runner records the verdict as `OOM`. +- Without the cap (the committed `baseline-c.tsv`), the peak is about + 12.8 GB, with UNKNOWN. + +**Cause: hypothesis, from reading the source; not profiled.** +- `helper_map2check_nondet_append_element` (`NonDetLog.c:96-103`) `malloc`s + one `NONDET_CALL` per call. `append_element` (`ContainerRealloc.c:44`) + copies it into a `realloc`ed array. The `malloc`ed row is never freed. +- `map2check_init`, which the pass puts at the start of the program's + `main`, runs once per `__AFL_LOOP(10000)` iteration + (`NonDetGeneratorAFL.c:171-184`). `nondet_log_init` replaces the container + with `new_container`, which sets `values = NULL` without freeing the + previous array. +- So every iteration leaks its whole nondet log twice over, and the + persistent-mode process grows by about 100 000 rows × 2 per iteration + until the cap. + +**Reproduce:** with the C runtime under `docker --memory 3g`: +`map2check --nondet-generator afl --target-function standard_sentinel-2.i` +(sv-benchmarks `array-examples`). The container's `memory.events` shows a +new `oom_kill`. The Rust runtime finishes with UNKNOWN and a peak of about +341 MB. + +```c +/* Every AFL++ iteration reads 100000 values. */ +int main(void) { + int a[100000]; + for (int i = 0; i < 100000; i++) + a[i] = __VERIFIER_nondet_int(); /* C: NonDetLog grows across iterations */ + return 0; +} +``` From ef166d1aa3a73059b181719fca08b8409b509f19 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 22:41:31 -0400 Subject: [PATCH 60/63] perf: stub std's backtrace symbolizer out of the KLEE runtime The panic hook drags gimli, addr2line, miniz_oxide, rustc_demangle and backtrace_rs into the fat-LTO module: 48% of its IR, which KLEE loads and re-optimises on every symex run. scripts/klee-stub-symbolizer.py turns their bodies (and std's backtrace printer) into abort stubs after --strip-debug, keeping bodies that rustc merged with code the runtime still reaches, and globaldce drops what only they used. A self-test fixture runs under lli-16 first and a guard fails the build if a symbolizer body is left. The module goes from 125829 to 52590 instruction lines and 2.3 MB to 1.2 MB; KLEE on byte_add_bug from 3.1 s / 221 MB to 1.3 s / 156 MB. llvm.bitreverse is gone, so klee-lower-intrinsics.py accepts no exception. tests/klee_paths.sh checks the size bound and that a panic still ends its path with a KLEE error, with and without RUST_BACKTRACE. --- .github/workflows/CI.yml | 5 +- docs/klee-feasibility.md | 6 +- scripts/build-klee-runtime.sh | 43 ++++++-- scripts/klee-lower-intrinsics.py | 11 +- scripts/klee-stub-symbolizer.py | 180 +++++++++++++++++++++++++++++++ tests/klee/panic.c | 18 ++++ tests/klee/panic_shim.py | 42 ++++++++ tests/klee/stub-symbolizer.ll | 51 +++++++++ tests/klee_paths.sh | 66 +++++++++++- 9 files changed, 404 insertions(+), 18 deletions(-) create mode 100644 scripts/klee-stub-symbolizer.py create mode 100644 tests/klee/panic.c create mode 100755 tests/klee/panic_shim.py create mode 100644 tests/klee/stub-symbolizer.ll diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index 5ce95cd..664f23e 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -195,7 +195,10 @@ jobs: # O script roda os autotestes da troca de intrínsecos antes de usá-la: # equivalência por lli-16 em tests/klee/lower-intrinsics.ll e recusa # de tests/klee/unknown-intrinsic.ll; qualquer intrínseco que o KLEE - # 3.1 não execute faz o build falhar. + # 3.1 não execute faz o build falhar. Antes da troca, o simbolizador + # de backtrace do std vira stubs de abort (scripts/klee-stub-symbolizer.py, + # com autoteste em tests/klee/stub-symbolizer.ll), e a guarda final + # falha se algum corpo do simbolizador sobrar. - name: Compilar o runtime KLEE em bitcode (klee/Cargo.toml, --locked) run: scripts/build-klee-runtime.sh target/klee/map2check-klee.bc diff --git a/docs/klee-feasibility.md b/docs/klee-feasibility.md index ce0b228..86ed70d 100644 --- a/docs/klee-feasibility.md +++ b/docs/klee-feasibility.md @@ -11,7 +11,7 @@ Data: 2026-10-16. Decisão de origem: D1, rota A′ (`docs/cutover/decisoes.md`) | Segundo manifesto, com `build = false`, sem dev-dependencies, `rust-version = "1.72"`, `[features] klee = []`, tetos de dependência e `[workspace]` próprio | `klee/Cargo.toml` | | Lockfile v3 próprio, gerado pelo resolvedor com fallback de MSRV (comando no cabeçalho do manifesto): serde 1.0.229, serde_json 1.0.154, thiserror 2.0.20 | `klee/Cargo.lock` | | Gerador KLEE: `__VERIFIER_nondet_` → `klee_make_symbolic`, `__VERIFIER_assume` → `klee_assume`, `main` → `__map2check_main__`, concretização antes da saída | `rust/src/nondet/klee.rs` | -| Build exato: rustc 1.72.1, `--emit=llvm-bc`, `panic = "abort"`, LTO fat, `codegen-units = 1`, `opt-16 --strip-debug`, troca dos intrínsecos vetoriais, leitura de volta com `llvm-dis-16` | `scripts/build-klee-runtime.sh`, `scripts/klee-lower-intrinsics.py` | +| Build exato: rustc 1.72.1, `--emit=llvm-bc`, `panic = "abort"`, LTO fat, `codegen-units = 1`, `opt-16 --strip-debug`, simbolizador de backtrace trocado por abort, troca dos intrínsecos vetoriais, leitura de volta com `llvm-dis-16` | `scripts/build-klee-runtime.sh`, `scripts/klee-stub-symbolizer.py`, `scripts/klee-lower-intrinsics.py` | | Teste de integração em Docker | `tests/klee_paths.sh`, `tests/klee/` (programas, `Dockerfile`, conversor ktest → semente) | | Job de CI em rustc 1.72.1 | `.github/workflows/CI.yml`, job `klee_runtime` | @@ -92,9 +92,9 @@ A semente tem esses 30 bytes concatenados. O mesmo `layout.c` foi ligado a `libm ## Achados e limitações -1. **Intrínsecos vetoriais do `std` pré-compilado.** O `std` do 1.72.1 vem vetorizado, e o LTO fat leva esse código para o módulo. O KLEE 3.1 não executa `llvm.vector.reduce.add/or` nem `llvm.x86.sse2.pmulhu.w`. O caminho de `deref.c` morria com `unimplemented intrinsic` em `core::fmt::Formatter::pad_integral`, ao formatar o log de heap em hexadecimal com largura. Desligar a vetorização no LTO (`-C no-vectorize-loops -C no-vectorize-slp`) não resolve, porque o código já vem vetorizado no bitcode do `std`. `scripts/klee-lower-intrinsics.py` troca cada intrínseco por uma função interna escalar (`extractelement`/`insertelement` e aritmética). Antes de usar a troca, o build roda dois autotestes: a equivalência com `lli-16` em `tests/klee/lower-intrinsics.ll` e a recusa de `tests/klee/unknown-intrinsic.ll` (`llvm.masked.load`). Depois da troca, qualquer intrínseco declarado que não esteja na lista dos que o KLEE 3.1 executa faz o build falhar. A lista vem dos `case Intrinsic::` de `Executor.cpp` e `IntrinsicCleaner.cpp` na tag v3.1, mais `bitreverse` como exceção justificada (item 3). O build escreve tudo num diretório temporário e só move o `.bc` no fim: uma falha não deixa bitcode no destino. Não usei o `opt-16 -enable-new-pm=0 -expand-reductions`: ele só cobre a família reduce (o script continuaria necessário para os intrínsecos x86) e vive no gerenciador de passes legado que o LLVM está removendo. Isto é uma sexta peça do produto, além das cinco da D1. +1. **Intrínsecos vetoriais do `std` pré-compilado.** O `std` do 1.72.1 vem vetorizado, e o LTO fat leva esse código para o módulo. O KLEE 3.1 não executa `llvm.vector.reduce.add/or` nem `llvm.x86.sse2.pmulhu.w`. O caminho de `deref.c` morria com `unimplemented intrinsic` em `core::fmt::Formatter::pad_integral`, ao formatar o log de heap em hexadecimal com largura. Desligar a vetorização no LTO (`-C no-vectorize-loops -C no-vectorize-slp`) não resolve, porque o código já vem vetorizado no bitcode do `std`. `scripts/klee-lower-intrinsics.py` troca cada intrínseco por uma função interna escalar (`extractelement`/`insertelement` e aritmética). Antes de usar a troca, o build roda dois autotestes: a equivalência com `lli-16` em `tests/klee/lower-intrinsics.ll` e a recusa de `tests/klee/unknown-intrinsic.ll` (`llvm.masked.load`). Depois da troca, qualquer intrínseco declarado que não esteja na lista dos que o KLEE 3.1 executa faz o build falhar. A lista vem dos `case Intrinsic::` de `Executor.cpp` e `IntrinsicCleaner.cpp` na tag v3.1, sem exceções desde a Task 2.8c (item 3). O build escreve tudo num diretório temporário e só move o `.bc` no fim: uma falha não deixa bitcode no destino. Não usei o `opt-16 -enable-new-pm=0 -expand-reductions`: ele só cobre a família reduce (o script continuaria necessário para os intrínsecos x86) e vive no gerenciador de passes legado que o LLVM está removendo. Isto é uma sexta peça do produto, além das cinco da D1. 2. **`RandomState` do `HashSet`.** O `RandomState::new()` busca a semente do SO por um símbolo fraco (`getrandom`). Sob o KLEE, isso vira `invalid function pointer`, e o caminho morre ao gravar os logs de witness. Os três `HashSet` do crate passaram a usar `memtrack::FixedHashSet`, com chaves SipHash fixas. Eles só respondem pertinência, então o resultado não muda em nenhum build. -3. **`llvm.bitreverse` (miniz_oxide).** Só a impressão de backtrace do `std` chega a ele, e com `panic = "abort"` ela não é executada. O KLEE avisa `unsupported intrinsic` no carregamento. Não dá para trocá-lo: o `--optimize` do KLEE refaz o intrínseco a partir da versão escalar. +3. **Simbolizador de backtrace do `std` (Task 2.8c).** O hook de pânico padrão do `std` pode imprimir um backtrace, e o LTO fat levava o simbolizador inteiro (gimli, addr2line, miniz_oxide, rustc_demangle, backtrace_rs) para o módulo, mesmo com `panic = "abort"`: 48% das linhas de IR, que o KLEE carregava e reotimizava em toda execução (cerca de 1,6 s e 60 MB por tarefa symex, critério 9 da Task 4.3). Depois do `--strip-debug`, `scripts/klee-stub-symbolizer.py` troca o corpo dessas funções e da impressora de backtrace (`std::sys_common::backtrace::print`, `_print_fmt`) por `call void @abort()` + `unreachable`, e o `opt-16 -passes=globaldce` remove o que só elas usavam. Nome e assinatura ficam, então nenhum símbolo fica indefinido, e um pânico com `RUST_BACKTRACE` ligado termina em abort, como sem ele. Corpos compartilhados ficam: o rustc funde funções idênticas, e um `drop_in_place>` é o mesmo que `alloc::fmt::format` chama; o script percorre o módulo a partir dos símbolos externos e só troca o que esse percurso não alcança sem passar pela impressora (dois corpos de 9 e 11 linhas ficam). Antes de usar a troca, o build roda o autoteste em `tests/klee/stub-symbolizer.ll` com `lli-16`; no fim, a guarda (`--check`) falha se sobrar corpo do simbolizador. Efeito: o módulo cai de 125 829 para 52 590 linhas de instrução e de 2,3 MB para 1,2 MB; o `llvm.bitreverse` do miniz_oxide sumiu, e a lista de exceções de `klee-lower-intrinsics.py` ficou vazia. `tests/klee_paths.sh` confere o limite de 80 000 linhas e que um pânico (`tests/klee/panic.c`, por `tests/klee/panic_shim.py`, só no teste) encerra o caminho com um erro do KLEE, com e sem `RUST_BACKTRACE`, sem executar código do simbolizador. Sob o KLEE 3.1 o hook de pânico do `std` morre antes de imprimir (`invalid function pointer` ou, com `RUST_BACKTRACE=full`, `illegal instruction` em `rust_panic_with_hook`): é um erro `exec`, não um abort, e já era assim antes dos stubs. 4. **Chamadas externas.** O `std` grava os arquivos por `open64`/`write`/`close`/`unlink` externos, como o `fopen`/`fprintf` do C. O KLEE avisa, por exemplo, `undefined reference to function: statx`, `pthread_key_create` e `_Unwind_*`. Nenhum desses símbolos é executado nos caminhos testados. 5. **`--external-calls=all` e o `std`.** O que usa símbolos fracos via `dlsym` ou `syscall!` (entropia, `statx`, TLS com destrutor) é inseguro sob o KLEE. Hoje o crate não usa nada disso no estado, mas qualquer código novo que entre no build `symex` precisa passar por `tests/klee_paths.sh`. 6. **Escopo dos testes.** Os programas imitam a instrumentação dos passes à mão (`tests/klee/m2c.h`). O pipeline real com os passes do Map2Check é a Task 3.x, e o `tests/klee_paths.sh` no CI do Map2Check é a Task 3.4. Overflow, memcleanup, `pchar` e `double` simbólicos não têm programa próprio. `layout.c` cobre os oito tipos inteiros. diff --git a/scripts/build-klee-runtime.sh b/scripts/build-klee-runtime.sh index 1993672..3d95538 100755 --- a/scripts/build-klee-runtime.sh +++ b/scripts/build-klee-runtime.sh @@ -6,9 +6,11 @@ # # rustc 1.72.1 (LLVM 16) on klee/Cargo.toml with its own Cargo.lock, # features `klee,wasm`, panic = abort, fat LTO, codegen-units = 1, -# --emit=llvm-bc, then `opt-16 --strip-debug` (condition 6), then the -# vector intrinsics KLEE cannot run lowered to scalar code -# (scripts/klee-lower-intrinsics.py), then a read-back with `llvm-dis-16`. +# --emit=llvm-bc, then `opt-16 --strip-debug` (condition 6), then std's +# backtrace symbolizer stubbed out (scripts/klee-stub-symbolizer.py, +# `opt-16 -passes=globaldce`), then the vector intrinsics KLEE cannot run +# lowered to scalar code (scripts/klee-lower-intrinsics.py), then a +# read-back with `llvm-dis-16` and the symbolizer guard. # # Usage: scripts/build-klee-runtime.sh # @@ -108,9 +110,35 @@ mapfile -t modules < <(find "$deps" -maxdepth 1 -name 'map2check-*.bc') # program keeps its -g. # Everything is built in $work and moved to $out only at the end, so a # failed step never leaves a bitcode behind. +fixtures="$ROOT/tests/klee" "$OPT" --strip-debug "${modules[0]}" -o "$work/stripped.bc" || die "$OPT --strip-debug failed" "$LLVM_DIS" "$work/stripped.bc" -o "$work/stripped.ll" || die "$LLVM_DIS cannot read the stripped module" +# std's backtrace symbolizer (gimli, addr2line, miniz_oxide, rustc_demangle, +# backtrace_rs) is about half of the module and only runs when a panic +# prints a backtrace; KLEE spent about 1.6 s and 60 MB per run loading and +# re-optimising it (task 4.3, criterion 9). Its bodies become a call to +# abort, so a path that reaches them still ends as an abort and no symbol +# is left undefined; globaldce then drops what only they used. The self-test runs first: a body +# the remaining code shares must keep working, the printer path must abort, +# and the guard must reject the fixture before the rewrite. +stub="$ROOT/scripts/klee-stub-symbolizer.py" +"$LLI" "$fixtures/stub-symbolizer.ll" >/dev/null 2>&1 && rc=0 || rc=$? +[ "$rc" = 42 ] || die "$LLI: the stub fixture returned $rc, not 42" +python3 -I "$stub" "$fixtures/stub-symbolizer.ll" "$work/stub-fixture.ll" >/dev/null || + die "the stub self-test fixture was rejected" +"$LLI" "$work/stub-fixture.ll" >/dev/null 2>&1 && rc=0 || rc=$? +[ "$rc" = 42 ] || die "the stub rewrite broke a shared body: the fixture returned $rc, not 42" +"$LLI" "$work/stub-fixture.ll" backtrace >/dev/null 2>&1 && rc=0 || rc=$? +[ "$rc" = 134 ] || die "the stubbed printer path did not abort: the fixture returned $rc, not 134" +if python3 -I "$stub" --check "$fixtures/stub-symbolizer.ll" >/dev/null 2>&1; then + die "the symbolizer guard accepted a module that still has the symbolizer (stub-symbolizer.ll)" +fi +python3 -I "$stub" "$work/stripped.ll" "$work/stubbed.ll" || die "stubbing the backtrace symbolizer failed" +"$LLVM_AS" "$work/stubbed.ll" -o "$work/stubbed.bc" || die "$LLVM_AS rejected the stubbed module" +"$OPT" -passes=globaldce "$work/stubbed.bc" -o "$work/dce.bc" || die "$OPT -passes=globaldce failed" +"$LLVM_DIS" "$work/dce.bc" -o "$work/dce.ll" || die "$LLVM_DIS cannot read the module after globaldce" + # KLEE 3.1 cannot execute the vector/x86 intrinsics of the precompiled std # ("unimplemented intrinsic" in core::fmt padding): replace them by scalar # functions. The script fails on any remaining intrinsic KLEE 3.1 does not @@ -118,7 +146,6 @@ mapfile -t modules < <(find "$deps" -maxdepth 1 -name 'map2check-*.bc') # what the intrinsics compute (lli-16), and an unknown intrinsic must be # rejected. lower="$ROOT/scripts/klee-lower-intrinsics.py" -fixtures="$ROOT/tests/klee" python3 -I "$lower" "$fixtures/lower-intrinsics.ll" "$work/fixture.ll" >/dev/null || die "the lowering self-test fixture was rejected" expected="$("$LLI" "$fixtures/lower-intrinsics.ll")" || die "$LLI failed on the fixture" @@ -128,10 +155,14 @@ got="$("$LLI" "$work/fixture.ll")" || die "$LLI failed on the lowered fixture" if python3 -I "$lower" "$fixtures/unknown-intrinsic.ll" "$work/unknown.ll" >/dev/null 2>&1; then die "the lowering accepted an intrinsic KLEE 3.1 cannot execute (unknown-intrinsic.ll)" fi -python3 -I "$lower" "$work/stripped.ll" "$work/lowered.ll" || +python3 -I "$lower" "$work/dce.ll" "$work/lowered.ll" || die "lowering the intrinsics failed" "$LLVM_AS" "$work/lowered.ll" -o "$work/runtime.bc" || die "$LLVM_AS rejected the lowered module" -"$LLVM_DIS" "$work/runtime.bc" -o /dev/null || die "$LLVM_DIS cannot read the lowered module" +"$LLVM_DIS" "$work/runtime.bc" -o "$work/runtime.ll" || die "$LLVM_DIS cannot read the lowered module" +# Guard: no symbolizer function that only the backtrace printer reaches may +# keep a body. The shared bodies it prints are kept on purpose (see the +# script). +python3 -I "$stub" --check "$work/runtime.ll" || die "the backtrace symbolizer is still in the module" # The module must define what the symex pipeline links against, and leave # the KLEE intrinsics to KLEE. diff --git a/scripts/klee-lower-intrinsics.py b/scripts/klee-lower-intrinsics.py index c662ff0..93c50fa 100644 --- a/scripts/klee-lower-intrinsics.py +++ b/scripts/klee-lower-intrinsics.py @@ -57,13 +57,10 @@ """.split() # Declared but accepted although KLEE 3.1 cannot run them, each with why it -# is never executed by the runtime. -ACCEPTED = { - # miniz_oxide, reached only by std's backtrace printing, which - # panic = "abort" never runs. KLEE warns "unsupported intrinsic" at - # load; KLEE's --optimize folds any scalar version back into it. - "bitreverse": "backtrace symbolisation only", -} +# is never executed by the runtime. Empty: `llvm.bitreverse` (miniz_oxide) +# was here until scripts/klee-stub-symbolizer.py removed the backtrace +# symbolizer, which was its only user, before this script runs. +ACCEPTED = {} def unsupported(names): diff --git a/scripts/klee-stub-symbolizer.py b/scripts/klee-stub-symbolizer.py new file mode 100644 index 0000000..10580c2 --- /dev/null +++ b/scripts/klee-stub-symbolizer.py @@ -0,0 +1,180 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-3.0-only +"""Replaces the body of every function of std's backtrace symbolizer in the +KLEE runtime's IR by a call to `abort`. + +std's default panic hook can print a backtrace, so fat LTO carries the whole +symbolizer (gimli, addr2line, miniz_oxide, rustc_demangle, backtrace_rs) into +the module, even with panic = "abort": about 48% of its IR, which KLEE loads, +links against uclibc and re-optimises (--optimize) on every run, about 1.6 s +and 60 MB per symex task (task 4.3, criterion 9). Only a panic with +RUST_BACKTRACE set reaches that code, and the runtime denies panics outside +tests (rust/src/lib.rs). + +Each stubbed function keeps its name and signature, so nothing that calls it +changes and no undefined symbol is introduced; its body becomes +`call void @abort()` + `unreachable`, which KLEE reports as an abort, the +same end a panic = "abort" panic has. Its function attributes are replaced +by `cold noinline noreturn nounwind`: an attribute such as `willreturn` or +`memory(none)` on a body that never returns would let the optimiser turn the +call sites into `unreachable` instead of an abort. `opt -passes=globaldce` +then drops what only those bodies used (the `object` crate, the +decompression tables). + +Shared bodies are kept. rustc merges identical functions, so one body named +after a symbolizer type (`drop_in_place>`) +is also the one `alloc::fmt::format` calls to free a `String`. The printer +entry points (PRINTER below) are always stubbed; any other candidate is +stubbed only if the code that stays cannot reach it: a walk from every +symbol with external linkage (`main`, the C API, `llvm.used`), through the +bodies and initialisers of everything it reaches except the printer, marks +the candidates it meets as shared, and those keep their bodies. + +Usage: + klee-stub-symbolizer.py rewrite + klee-stub-symbolizer.py --check fail if a candidate that only + symbolizer code reaches still + has a real body""" +import re +import sys + +# A candidate is a function whose (mangled) name names one of these crates +# or std's copy of backtrace-rs ... +SYMBOLIZER = re.compile(r"gimli|addr2line|miniz_oxide|rustc_demangle|backtrace_rs") +# ... or std's backtrace printer, which calls the symbolizer and is reached +# only from the panic hook when a backtrace is asked for. The short-backtrace +# markers (`__rust_{begin,end}_short_backtrace`), on every panic path, are +# not candidates. +PRINTER = re.compile( + r"sys_common9backtrace5print|sys_common9backtrace10_print_fmt" + r"|sys_common9backtrace15output_filename|sys_common\.\.backtrace\.\._print" +) + +NAME = r'@(?:"(?P(?:[^"\\]|\\.)*)"|(?P[-\w.$]+))' +DEFINE = re.compile(r"^define [^\n]*?" + NAME + r"\(", re.M) +GLOBAL = re.compile(r"^" + NAME + r" = [^\n]*$", re.M) +REF = re.compile(NAME) +LOCAL_DEFINE = re.compile(r"define (?:internal|private) ") +LOCAL_GLOBAL = re.compile(r'"? = (?:internal|private) ') +STUB_ATTRS = "cold noinline noreturn nounwind" +STUB_BODY = [" call void @abort()", " unreachable"] + + +def name_of(m): + return m["q"] if m["q"] is not None else m["n"] + + +def candidate(name): + return bool(SYMBOLIZER.search(name) or PRINTER.search(name)) + + +def parse(ir): + """Returns the definitions as {name: (start, end)} (`end` just past the + closing `}` line), the names each global symbol (function body, variable + or alias initialiser) refers to, and the roots: the symbols with + external linkage.""" + defs, refs, roots = {}, {}, set() + for m in DEFINE.finditer(ir): + name = name_of(m) + end = ir.index("\n}\n", m.start()) + 3 + defs[name] = (m.start(), end) + body = ir[ir.index("\n", m.start()) : end] + refs[name] = {name_of(r) for r in REF.finditer(body)} + if not LOCAL_DEFINE.match(ir, m.start()): + roots.add(name) + for m in GLOBAL.finditer(ir): + name = name_of(m) + rest = m.group(0)[m.end("q" if m["q"] is not None else "n") :] + refs[name] = {name_of(r) for r in REF.finditer(rest)} + if not LOCAL_GLOBAL.match(rest): + roots.add(name) + return defs, refs, roots + + +def stub_set(defs, refs, roots): + """The candidates the code that stays cannot reach: a walk from the + roots that does not enter the printer.""" + seen, todo = set(), [r for r in roots if not PRINTER.search(r)] + while todo: + name = todo.pop() + if name in seen: + continue + seen.add(name) + todo.extend(r for r in refs.get(name, ()) if r not in seen and not PRINTER.search(r)) + return {n for n in defs if candidate(n) and n not in seen} + + +def stub_header(header, group): + header = re.sub(r" #\d+(?= |$)", "", header) + header = re.sub(r" personality ptr " + NAME, "", header) + if not header.endswith(" {"): + sys.exit(f"klee-stub-symbolizer: unexpected definition line: {header[:200]}") + return header[: -len(" {")] + f" #{group} {{" + + +def rewrite(ir): + defs, refs, roots = parse(ir) + stub = stub_set(defs, refs, roots) + groups = [int(g) for g in re.findall(r"^attributes #(\d+) ", ir, re.M)] + group = max(groups, default=-1) + 1 + out, last = [], 0 + for name, (start, end) in sorted(defs.items(), key=lambda d: d[1][0]): + if name not in stub: + continue + header = ir[start : ir.index("\n", start)] + out.append(ir[last:start]) + out.append("\n".join([stub_header(header, group), "start:"] + STUB_BODY + ["}"]) + "\n") + last = end + out.append(ir[last:]) + ir = "".join(out) + if stub: + if not re.search(r"^declare [^\n]*@abort\(\)", ir, re.M): + ir += "\ndeclare void @abort()\n" + ir += f"\nattributes #{group} = {{ {STUB_ATTRS} }}\n" + kept = sorted(n for n in defs if candidate(n) and n not in stub) + return ir, len(stub), kept + + +def is_stub(ir, start, end): + body = [line.strip() for line in ir[start:end].splitlines()[1:-1]] + return [b for b in body if b and not b.endswith(":")] == [s.strip() for s in STUB_BODY] + + +def check(ir): + """The candidates that only symbolizer code reaches but that still have + a real body, and the kept (shared) candidates with their sizes.""" + defs, refs, roots = parse(ir) + stub = stub_set(defs, refs, roots) + bad = sorted(n for n in stub if not is_stub(ir, *defs[n])) + kept = { + n: ir[s:e].count("\n") - 2 + for n, (s, e) in defs.items() + if candidate(n) and n not in stub and not is_stub(ir, s, e) + } + return bad, kept + + +def main(): + if len(sys.argv) == 3 and sys.argv[1] == "--check": + with open(sys.argv[2]) as f: + bad, kept = check(f.read()) + if bad: + sys.exit( + "klee-stub-symbolizer: backtrace symbolizer code is still in the module: " + + ", ".join(bad[:10]) + + (f" (and {len(bad) - 10} more)" if len(bad) > 10 else "") + ) + for name, size in sorted(kept.items()): + print(f"klee-stub-symbolizer: shared body kept ({size} lines): {name}") + return + if len(sys.argv) != 3 or sys.argv[1].startswith("-"): + sys.exit(__doc__) + with open(sys.argv[1]) as f: + ir, stubbed, kept = rewrite(f.read()) + with open(sys.argv[2], "w") as f: + f.write(ir) + print(f"klee-stub-symbolizer: {stubbed} symbolizer function(s) now abort, {len(kept)} shared kept") + + +if __name__ == "__main__": + main() diff --git a/tests/klee/panic.c b/tests/klee/panic.c new file mode 100644 index 0000000..17e5f18 --- /dev/null +++ b/tests/klee/panic.c @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-3.0-only + * A Rust panic under KLEE, after the backtrace symbolizer was stubbed out + * (scripts/klee-stub-symbolizer.py): x == 7 panics (index out of bounds, + * through tests/klee/panic_shim.py, test only), which must end its path + * with a KLEE error and never run symbolizer code, with or without + * RUST_BACKTRACE; the other side ends normally. */ +#include "m2c.h" + +void m2c_test_panic(long index); + +int __map2check_main__(void) { + M2C_START(M2C_MODE_TARGET); + int x = NONDET(int, int, 12); + if (x == 7) + m2c_test_panic(x); + map2check_success(); + return 0; +} diff --git a/tests/klee/panic_shim.py b/tests/klee/panic_shim.py new file mode 100755 index 0000000..1a422c6 --- /dev/null +++ b/tests/klee/panic_shim.py @@ -0,0 +1,42 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-3.0-only +"""Test only: gives a copy of the KLEE runtime's IR a C entry point that +panics, `void m2c_test_panic(long index)`, so tests/klee_paths.sh can check +that a panic still ends its path (a KLEE error, not a hang) after the backtrace +symbolizer was stubbed out (scripts/klee-stub-symbolizer.py). The runtime +denies panics outside tests, so no C API function reaches one. + +The entry point calls `core::panicking::panic_bounds_check(index, 4, loc)` +with the `Location` of an existing call site: the real path through the +panic handler, std's default hook (which prints a backtrace when +RUST_BACKTRACE is set) and panic = "abort". + +Usage: panic_shim.py """ +import re +import sys + +CALL = re.compile( + r"call fastcc void @(?P_ZN4core9panicking18panic_bounds_check17h[0-9a-f]+E)" + r"\(i64 [^,]*, i64 [^,]*, ptr [^@%]*(?P@[-\w.$]+)\)" +) + + +def main(): + if len(sys.argv) != 3: + sys.exit(__doc__) + with open(sys.argv[1]) as f: + ir = f.read() + m = CALL.search(ir) + if not m: + sys.exit("panic_shim: no call to core::panicking::panic_bounds_check in the module") + with open(sys.argv[2], "w") as f: + f.write(ir) + f.write( + "\ndefine void @m2c_test_panic(i64 %i) {\n" + f" call fastcc void @{m['f']}(i64 %i, i64 4, ptr {m['loc']})\n" + " unreachable\n}\n" + ) + + +if __name__ == "__main__": + main() diff --git a/tests/klee/stub-symbolizer.ll b/tests/klee/stub-symbolizer.ll new file mode 100644 index 0000000..ef147f5 --- /dev/null +++ b/tests/klee/stub-symbolizer.ll @@ -0,0 +1,51 @@ +; SPDX-License-Identifier: GPL-3.0-only +; Self-test of scripts/klee-stub-symbolizer.py, run by +; scripts/build-klee-runtime.sh (and so by the CI job klee_runtime) with +; lli-16 before and after the rewrite: +; - no argument: main returns 42 through @shared, a body named after a +; gimli type that the remaining code also calls (rustc merges identical +; functions); it must keep its body, so 42 before and after; +; - one argument: main calls the backtrace printer, which calls the +; symbolizer; 42 before, an abort (SIGABRT) after; +; - `--check` rejects this file as it is and accepts the rewritten one. +@vtable = internal constant [1 x ptr] [ptr @_ZN3std10sys_common9backtrace5print17h0000000000000001E] + +define internal i32 @_ZN5gimli4read5parse17h0000000000000002E(i32 %x) unnamed_addr #0 personality ptr @rust_eh_personality { +start: + %y = mul i32 %x, 2 + %z = sub i32 %y, 42 + ret i32 %z +} + +define internal i32 @_ZN3std10sys_common9backtrace5print17h0000000000000001E(i32 %x) unnamed_addr #0 { +start: + %r = call i32 @_ZN5gimli4read5parse17h0000000000000002E(i32 %x) + ret i32 %r +} + +define internal i32 @"_ZN4core3ptr13drop_in_place$LT$gimli..Mapping$GT$17h0000000000000003E"(i32 %x) unnamed_addr #0 { +start: + %y = add i32 %x, 1 + ret i32 %y +} + +define i32 @rust_eh_personality() { + ret i32 0 +} + +define i32 @main(i32 %argc, ptr %argv) { +start: + %a = call i32 @"_ZN4core3ptr13drop_in_place$LT$gimli..Mapping$GT$17h0000000000000003E"(i32 41) + %bt = icmp sgt i32 %argc, 1 + br i1 %bt, label %print, label %done + +print: + %f = load ptr, ptr @vtable + %p = call i32 %f(i32 %a) + ret i32 %p + +done: + ret i32 %a +} + +attributes #0 = { nounwind willreturn memory(none) } diff --git a/tests/klee_paths.sh b/tests/klee_paths.sh index 1a2159a..4938769 100755 --- a/tests/klee_paths.sh +++ b/tests/klee_paths.sh @@ -72,6 +72,8 @@ fail() { # --max-time keeps a path blow-up from hanging the test. KLEE_FLAGS=(--external-calls=all --optimize --use-cex-cache --solver-backend=z3 --libc=uclibc --max-time=300s) +# Extra environment of KLEE, which passes its own to the program. +KLEE_ENV=(RUST_BACKTRACE=0) # The lowering's self-tests (lli-16 equivalence on tests/klee/ # lower-intrinsics.ll, rejection of tests/klee/unknown-intrinsic.ll) run @@ -126,6 +128,8 @@ run_klee() { cflags=(-DM2C_DEVELOP) rtbc="$W/dev-$mode.bc" [ -f "$rtbc" ] || develop_runtime "$mode" "$rtbc" + elif [ "$rt" = rustpanic ]; then + rtbc="$W/rust-rt-panic.bc" else rtbc="$W/rust-rt.bc" fi @@ -139,8 +143,12 @@ run_klee() { llvm-link-16 "${progs[@]}" "$rtbc" -o "$dir/linked.bc" local extra=() [ "$kind" = frontend ] && extra=(--exit-on-error-type=Abort) - (cd "$dir" && klee "${KLEE_FLAGS[@]}" "${extra[@]}" --output-dir=klee-out linked.bc \ + local start end + start="$(date +%s%N)" + (cd "$dir" && env "${KLEE_ENV[@]}" klee "${KLEE_FLAGS[@]}" "${extra[@]}" --output-dir=klee-out linked.bc \ >klee.log 2>&1) || true + end="$(date +%s%N)" + KLEE_WALL_MS=$(((end - start) / 1000000)) PATHS_COMPLETED="$(sed -n 's/.*KLEE: done: completed paths = \([0-9]*\).*/\1/p' "$dir/klee.log")" PATHS_PARTIAL="$(sed -n 's/.*KLEE: done: partially completed paths = \([0-9]*\).*/\1/p' "$dir/klee.log")" PATHS_COMPLETED="${PATHS_COMPLETED:-?}" @@ -166,6 +174,7 @@ count() { COUNT["$1/$2"]="$PATHS_COMPLETED+$PATHS_PARTIAL"; } echo "== prog.c" run_klee prog rust full count prog rust +PROG_WALL_MS="$KLEE_WALL_MS" if [ "$PATHS_COMPLETED" = 2 ]; then pass "prog.c: KLEE: done: completed paths = 2" else @@ -410,6 +419,61 @@ else fail "layout.c: no seed from KLEE" fi +# ── the backtrace symbolizer is out of the module (task 2.8c) ──────────────── +# scripts/klee-stub-symbolizer.py turns std's symbolizer into abort stubs +# and the build's guard checks it. The bound is on the IR KLEE has to load +# and re-optimise, not on time: the module had 125 829 instruction lines +# with the symbolizer and about 52 600 without; KLEE's start-up follows it +# (3.1 s -> about 1.5 s on byte_add_bug), but a wall-clock bound would fail +# on a loaded host. +echo "== the KLEE runtime without the backtrace symbolizer" +llvm-dis-16 "$W/rust-rt.bc" -o "$W/rust-rt.ll" +lines="$(awk '/^define/{f=1; next} f&&/^}/{f=0; next} f{n++} END{print n+0}' "$W/rust-rt.ll")" +if [ "$lines" -lt 80000 ]; then + pass "the runtime module has $lines instruction lines (< 80000; 125829 with the symbolizer)" +else + fail "the runtime module has $lines instruction lines, expected < 80000: is the symbolizer back?" +fi +if python3 -I "$ROOT/scripts/klee-stub-symbolizer.py" --check "$W/rust-rt.ll" >/dev/null; then + pass "no symbolizer function that only the backtrace printer reaches has a body" +else + fail "the backtrace symbolizer is in the module" +fi +if grep -q 'llvm\.bitreverse' "$W/rust-rt.ll"; then + fail "llvm.bitreverse (miniz_oxide) is still declared" +else + pass "llvm.bitreverse, which only miniz_oxide used, is gone" +fi + +# A panic must still end the path with a KLEE error (no hang, no +# symbolizer code run), with or without RUST_BACKTRACE, which sends std's +# default hook toward the stubbed printer. tests/klee/panic_shim.py gives a +# copy of the runtime a C entry point that panics. Under KLEE 3.1 std's +# panic hook dies before printing anything, as it did before the stubs +# (task 2.8c): "invalid function pointer" (RUST_BACKTRACE=0) or "illegal +# instruction" (full) in rust_panic_with_hook, an exec error rather than an +# abort; the test accepts any error there and checks it is the x = 7 path. +echo "== panic.c" +python3 -I "$T/panic_shim.py" "$W/rust-rt.ll" "$W/rust-rt-panic.ll" +llvm-as-16 "$W/rust-rt-panic.ll" -o "$W/rust-rt-panic.bc" +for bt in 0 full; do + KLEE_ENV=(RUST_BACKTRACE="$bt") + run_klee panic rustpanic full + d="$W/panic-rustpanic-full" + err="$(find "$d/klee-out" -name '*.err' | sort | head -n 1)" + if [ "$PATHS_COMPLETED" = 1 ] && [ "$PATHS_PARTIAL" = 1 ] && [ -n "$err" ] && + ktest_objects "${err%.*.err}.ktest" "$W/seed.tmp" | grep -qx 'non_det_int 4 07000000' && + grep -q 'panicking' "$err" && + ! grep -Eq 'gimli|addr2line|miniz_oxide|rustc_demangle|backtrace_rs' "$err"; then + pass "panic.c (RUST_BACKTRACE=$bt): the panic (x = 7) ends its path with a KLEE error in std's panic code ($(head -n 1 "$err")); the other path completes" + else + fail "panic.c (RUST_BACKTRACE=$bt): $PATHS_COMPLETED+$PATHS_PARTIAL paths, error '$err', see $d/klee.log" + fi + mv "$d" "$d-$bt" +done +KLEE_ENV=(RUST_BACKTRACE=0) +echo "KLEE wall on prog.c (frontend flags, information only): ${PROG_WALL_MS:-?} ms" + # ── develop's C runtime, same programs ──────────────────────────────────────── if [ -n "${DEVELOP_SRC:-}" ]; then echo "== develop C runtime ($DEVELOP_SRC)" From 94a17ec28ec8de78288e3775a81ca22a6b02b248 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Fri, 16 Oct 2026 22:51:27 -0400 Subject: [PATCH 61/63] fix: stop the symbolizer walk only at the backtrace printer's entry klee-stub-symbolizer.py never walked into, and always stubbed, any function matching the printer pattern. That pattern also matches the drop_in_place of a _print_fmt closure, a plain buffer drop rustc may merge with a String drop; had live code called it, every format! would abort under KLEE, and --check, which ran the same walk, would not have seen it. The walk now stops only at std::sys_common::backtrace::print, the call the default panic hook makes when a backtrace is asked for; every other printer function is an ordinary candidate, kept when live code reaches it. --check also fails when live code reaches a stub. The fixture gains a printer-named drop that main calls (the old script stubbed it: lli returned 134), and stub-symbolizer-live.ll must be rejected by the guard. The runtime module is byte-identical: 142 stubbed, 2 kept, 52590 lines. --- scripts/build-klee-runtime.sh | 10 ++++- scripts/klee-stub-symbolizer.py | 65 ++++++++++++++++++------------ tests/klee/stub-symbolizer-live.ll | 21 ++++++++++ tests/klee/stub-symbolizer.ll | 20 ++++++--- 4 files changed, 84 insertions(+), 32 deletions(-) create mode 100644 tests/klee/stub-symbolizer-live.ll diff --git a/scripts/build-klee-runtime.sh b/scripts/build-klee-runtime.sh index 3d95538..fa9b999 100755 --- a/scripts/build-klee-runtime.sh +++ b/scripts/build-klee-runtime.sh @@ -120,8 +120,9 @@ fixtures="$ROOT/tests/klee" # re-optimising it (task 4.3, criterion 9). Its bodies become a call to # abort, so a path that reaches them still ends as an abort and no symbol # is left undefined; globaldce then drops what only they used. The self-test runs first: a body -# the remaining code shares must keep working, the printer path must abort, -# and the guard must reject the fixture before the rewrite. +# the remaining code shares must keep working (a printer-named one too), +# the printer path must abort, and the guard must reject the fixture before +# the rewrite and a stub that live code calls. stub="$ROOT/scripts/klee-stub-symbolizer.py" "$LLI" "$fixtures/stub-symbolizer.ll" >/dev/null 2>&1 && rc=0 || rc=$? [ "$rc" = 42 ] || die "$LLI: the stub fixture returned $rc, not 42" @@ -134,6 +135,11 @@ python3 -I "$stub" "$fixtures/stub-symbolizer.ll" "$work/stub-fixture.ll" >/dev/ if python3 -I "$stub" --check "$fixtures/stub-symbolizer.ll" >/dev/null 2>&1; then die "the symbolizer guard accepted a module that still has the symbolizer (stub-symbolizer.ll)" fi +python3 -I "$stub" --check "$work/stub-fixture.ll" >/dev/null || + die "the symbolizer guard rejected the rewritten fixture" +if python3 -I "$stub" --check "$fixtures/stub-symbolizer-live.ll" >/dev/null 2>&1; then + die "the symbolizer guard accepted a stub that live code calls (stub-symbolizer-live.ll)" +fi python3 -I "$stub" "$work/stripped.ll" "$work/stubbed.ll" || die "stubbing the backtrace symbolizer failed" "$LLVM_AS" "$work/stubbed.ll" -o "$work/stubbed.bc" || die "$LLVM_AS rejected the stubbed module" "$OPT" -passes=globaldce "$work/stubbed.bc" -o "$work/dce.bc" || die "$OPT -passes=globaldce failed" diff --git a/scripts/klee-stub-symbolizer.py b/scripts/klee-stub-symbolizer.py index 10580c2..c985616 100644 --- a/scripts/klee-stub-symbolizer.py +++ b/scripts/klee-stub-symbolizer.py @@ -23,32 +23,40 @@ Shared bodies are kept. rustc merges identical functions, so one body named after a symbolizer type (`drop_in_place>`) -is also the one `alloc::fmt::format` calls to free a `String`. The printer -entry points (PRINTER below) are always stubbed; any other candidate is -stubbed only if the code that stays cannot reach it: a walk from every +is also the one `alloc::fmt::format` calls to free a `String`. A candidate +is stubbed only if the code that stays cannot reach it: a walk from every symbol with external linkage (`main`, the C API, `llvm.used`), through the -bodies and initialisers of everything it reaches except the printer, marks -the candidates it meets as shared, and those keep their bodies. +bodies and initialisers of everything it reaches, stopping only at the +printer's entry (ENTRY below, the call std's default hook makes when a +backtrace is asked for), marks the candidates it meets as shared, and those +keep their bodies. Usage: klee-stub-symbolizer.py rewrite klee-stub-symbolizer.py --check fail if a candidate that only symbolizer code reaches still - has a real body""" + has a real body, or if one that + live code reaches is a stub""" import re import sys # A candidate is a function whose (mangled) name names one of these crates # or std's copy of backtrace-rs ... SYMBOLIZER = re.compile(r"gimli|addr2line|miniz_oxide|rustc_demangle|backtrace_rs") -# ... or std's backtrace printer, which calls the symbolizer and is reached -# only from the panic hook when a backtrace is asked for. The short-backtrace -# markers (`__rust_{begin,end}_short_backtrace`), on every panic path, are -# not candidates. +# ... or std's backtrace printer, which calls the symbolizer. The +# short-backtrace markers (`__rust_{begin,end}_short_backtrace`), on every +# panic path, are not candidates. PRINTER = re.compile( r"sys_common9backtrace5print|sys_common9backtrace10_print_fmt" r"|sys_common9backtrace15output_filename|sys_common\.\.backtrace\.\._print" ) +# The one edge the walk does not follow: `std::sys_common::backtrace::print`, +# the printer's entry, which std's default panic hook calls only when a +# backtrace is asked for. Every other printer function (its closures, the +# `drop_in_place` of its closure, which is a plain buffer drop rustc may +# merge with a `String` drop) is an ordinary candidate, kept when the walk +# reaches it. +ENTRY = re.compile(r"^_ZN3std10sys_common9backtrace5print17h[0-9a-f]{16}E$") NAME = r'@(?:"(?P(?:[^"\\]|\\.)*)"|(?P[-\w.$]+))' DEFINE = re.compile(r"^define [^\n]*?" + NAME + r"\(", re.M) @@ -92,16 +100,18 @@ def parse(ir): def stub_set(defs, refs, roots): - """The candidates the code that stays cannot reach: a walk from the - roots that does not enter the printer.""" - seen, todo = set(), [r for r in roots if not PRINTER.search(r)] + """The candidates the code that stays cannot reach (a walk from the + roots that does not enter the printer's entry), and the candidates it + reaches.""" + seen, todo = set(), [r for r in roots if not ENTRY.match(r)] while todo: name = todo.pop() if name in seen: continue seen.add(name) - todo.extend(r for r in refs.get(name, ()) if r not in seen and not PRINTER.search(r)) - return {n for n in defs if candidate(n) and n not in seen} + todo.extend(r for r in refs.get(name, ()) if r not in seen and not ENTRY.match(r)) + stub = {n for n in defs if candidate(n) and n not in seen} + return stub, {n for n in defs if candidate(n) and n in seen} def stub_header(header, group): @@ -114,7 +124,7 @@ def stub_header(header, group): def rewrite(ir): defs, refs, roots = parse(ir) - stub = stub_set(defs, refs, roots) + stub, _ = stub_set(defs, refs, roots) groups = [int(g) for g in re.findall(r"^attributes #(\d+) ", ir, re.M)] group = max(groups, default=-1) + 1 out, last = [], 0 @@ -142,28 +152,33 @@ def is_stub(ir, start, end): def check(ir): """The candidates that only symbolizer code reaches but that still have - a real body, and the kept (shared) candidates with their sizes.""" + a real body, the candidates that live code reaches but that are stubs + (live code would abort), and the kept (shared) candidates with their + sizes.""" defs, refs, roots = parse(ir) - stub = stub_set(defs, refs, roots) + stub, live = stub_set(defs, refs, roots) bad = sorted(n for n in stub if not is_stub(ir, *defs[n])) - kept = { - n: ir[s:e].count("\n") - 2 - for n, (s, e) in defs.items() - if candidate(n) and n not in stub and not is_stub(ir, s, e) - } - return bad, kept + dead = sorted(n for n in live if is_stub(ir, *defs[n])) + kept = {n: ir[defs[n][0] : defs[n][1]].count("\n") - 2 for n in live if n not in dead} + return bad, dead, kept def main(): if len(sys.argv) == 3 and sys.argv[1] == "--check": with open(sys.argv[2]) as f: - bad, kept = check(f.read()) + bad, dead, kept = check(f.read()) if bad: sys.exit( "klee-stub-symbolizer: backtrace symbolizer code is still in the module: " + ", ".join(bad[:10]) + (f" (and {len(bad) - 10} more)" if len(bad) > 10 else "") ) + if dead: + sys.exit( + "klee-stub-symbolizer: code outside the backtrace printer calls an abort stub: " + + ", ".join(dead[:10]) + + (f" (and {len(dead) - 10} more)" if len(dead) > 10 else "") + ) for name, size in sorted(kept.items()): print(f"klee-stub-symbolizer: shared body kept ({size} lines): {name}") return diff --git a/tests/klee/stub-symbolizer-live.ll b/tests/klee/stub-symbolizer-live.ll new file mode 100644 index 0000000..0578e49 --- /dev/null +++ b/tests/klee/stub-symbolizer-live.ll @@ -0,0 +1,21 @@ +; SPDX-License-Identifier: GPL-3.0-only +; Self-test of the guard of scripts/klee-stub-symbolizer.py, run by +; scripts/build-klee-runtime.sh: main (live code) calls a body named after +; std's backtrace printer that is an abort stub. If rustc merged that drop +; with a `String` drop, every format! would abort under KLEE, so `--check` +; must reject this module. +define internal i32 @"_ZN4core3ptr88drop_in_place$LT$std..sys_common..backtrace.._print_fmt..$u7b$$u7b$closure$u7d$$u7d$$GT$17h0000000000000004E"(i32 %x) unnamed_addr #0 { +start: + call void @abort() + unreachable +} + +define i32 @main() { +start: + %a = call i32 @"_ZN4core3ptr88drop_in_place$LT$std..sys_common..backtrace.._print_fmt..$u7b$$u7b$closure$u7d$$u7d$$GT$17h0000000000000004E"(i32 42) + ret i32 %a +} + +declare void @abort() + +attributes #0 = { cold noinline noreturn nounwind } diff --git a/tests/klee/stub-symbolizer.ll b/tests/klee/stub-symbolizer.ll index ef147f5..1e5ed69 100644 --- a/tests/klee/stub-symbolizer.ll +++ b/tests/klee/stub-symbolizer.ll @@ -2,12 +2,16 @@ ; Self-test of scripts/klee-stub-symbolizer.py, run by ; scripts/build-klee-runtime.sh (and so by the CI job klee_runtime) with ; lli-16 before and after the rewrite: -; - no argument: main returns 42 through @shared, a body named after a -; gimli type that the remaining code also calls (rustc merges identical -; functions); it must keep its body, so 42 before and after; +; - no argument: main returns 42 through two bodies named after +; symbolizer and printer types that the remaining code also calls +; (rustc merges identical functions: a gimli drop and the drop of a +; `_print_fmt` closure); both must keep their bodies, so 42 before and +; after; ; - one argument: main calls the backtrace printer, which calls the ; symbolizer; 42 before, an abort (SIGABRT) after; -; - `--check` rejects this file as it is and accepts the rewritten one. +; - `--check` rejects this file as it is and accepts the rewritten one; +; it also rejects tests/klee/stub-symbolizer-live.ll, where the +; printer-named drop main calls is a stub. @vtable = internal constant [1 x ptr] [ptr @_ZN3std10sys_common9backtrace5print17h0000000000000001E] define internal i32 @_ZN5gimli4read5parse17h0000000000000002E(i32 %x) unnamed_addr #0 personality ptr @rust_eh_personality { @@ -29,13 +33,19 @@ start: ret i32 %y } +define internal i32 @"_ZN4core3ptr88drop_in_place$LT$std..sys_common..backtrace.._print_fmt..$u7b$$u7b$closure$u7d$$u7d$$GT$17h0000000000000004E"(i32 %x) unnamed_addr #0 { +start: + ret i32 %x +} + define i32 @rust_eh_personality() { ret i32 0 } define i32 @main(i32 %argc, ptr %argv) { start: - %a = call i32 @"_ZN4core3ptr13drop_in_place$LT$gimli..Mapping$GT$17h0000000000000003E"(i32 41) + %a0 = call i32 @"_ZN4core3ptr13drop_in_place$LT$gimli..Mapping$GT$17h0000000000000003E"(i32 41) + %a = call i32 @"_ZN4core3ptr88drop_in_place$LT$std..sys_common..backtrace.._print_fmt..$u7b$$u7b$closure$u7d$$u7d$$GT$17h0000000000000004E"(i32 %a0) %bt = icmp sgt i32 %argc, 1 br i1 %bt, label %print, label %done From 511fb6e618ed7181e40b025227bb84f0ce756d74 Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Sat, 17 Oct 2026 01:16:50 -0400 Subject: [PATCH 62/63] docs: record D2 as approved --- docs/cutover/decisoes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cutover/decisoes.md b/docs/cutover/decisoes.md index f62e24f..8f4380d 100644 --- a/docs/cutover/decisoes.md +++ b/docs/cutover/decisoes.md @@ -157,7 +157,7 @@ A aprovação aceita a proposta junto com estas condições. **Pergunta:** o modo `--wasm` entra no cutover? -**Status: recomendação do plano, executada.** O plano registra a D2 só como recomendação ("Portar"), sem aprovação explícita do mantenedor. As Tasks 2.8 e 2.8b a seguiram. Falta a aprovação formal. +**Status: APROVADA em 2026-10-16** pelo mantenedor, depois de executada nas Tasks 2.8, 2.8b e 3.3 a partir da recomendação do plano ("Portar"). **Decisão (texto do plano):** portar os stubs do wasm2c para Rust, e não deixar `--wasm` fora do escopo. From 3f4f40b74a712a222e250a8ac64632ae21d5752b Mon Sep 17 00:00:00 2001 From: Herbert Rocha Date: Sat, 17 Oct 2026 01:26:37 -0400 Subject: [PATCH 63/63] build: copy the files the tests read into the Docker test stage The golden tests read tests/legacy_golden and the integration tests read docs/, CHANGELOG.md and the Dockerfile itself, none of which the test stage copied, so the CI docker build failed since 1.1.0-rc1. --- Dockerfile | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Dockerfile b/Dockerfile index a49a469..5d27311 100644 --- a/Dockerfile +++ b/Dockerfile @@ -28,6 +28,11 @@ RUN mkdir -p include && cargo build --release FROM builder AS test COPY test/ ./test/ +# Os testes leem os goldens do runtime C, os docs de compatibilidade, o +# CHANGELOG, o mapa de ABI e o próprio Dockerfile (label de versão). +COPY tests/ ./tests/ +COPY docs/ ./docs/ +COPY CHANGELOG.md Dockerfile ./ COPY scripts/coverage.sh ./scripts/coverage.sh RUN cargo test --all-features 2>&1 | tee /tmp/test-results.txt