Skip to content

DAOS-18626 build: Add ASan/UBSan/TSan compiler and Docker build support - #18613

Merged
daltonbohning merged 12 commits into
masterfrom
ckochhof/dev/master/daos-18626/patch-002
Sep 16, 2026
Merged

daltonbohning merged 12 commits into
masterfrom
ckochhof/dev/master/daos-18626/patch-002

Conversation

@knard38

@knard38 knard38 commented Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

Description

This is PR 1 of 10 in the split of #18557 (see ckochhof/dev/master/daos-18626/patch-001 for the original, unsplit commit). It adds the build-system plumbing needed to compile DAOS with UndefinedBehaviorSanitizer and ThreadSanitizer, in addition to the already-supported AddressSanitizer. It does not add any CI wiring or test-runner changes — those land in later PRs in this stack.

What/why

  • site_scons/site_tools/compiler_setup.py: link the UBSan/TSan runtime libraries (-lubsan/-ltsan) explicitly, including into CGO_LDFLAGS so that go build picks them up when CGO-linking a sanitizer-compiled C archive (otherwise Go binaries get "DSO missing from command line" for libubsan.so.1).
  • src/control/SConscript: skip control man-page generation under a SANITIZERS build (not needed, avoids extra build cost).
  • src/gurt/misc.c: skip d_free()'s debug poisoning memset() under TSan, which otherwise turns a harmless zero-size-allocation address reuse into a false heap-use-after-free report (confirmed via an isolated single-threaded reproducer to be a TSan tool limitation, not a DAOS bug).
  • utils/scripts/install-el9.sh: install compiler-rt, libubsan, and libtsan alongside the existing libasan dependency.
  • utils/docker/Dockerfile.el.9: thread a new SANITIZERS build-arg through to the scons invocation (empty/unset by default, so existing callers are unaffected).

This is purely additive: no existing (non-sanitizer) build is affected, and SANITIZERS=address behavior is unchanged.

Steps for the author:

  • Commit message follows the guidelines.
  • Appropriate Features or Test-tag pragmas were used.
  • Appropriate Functional Test Stages were run.
  • At least two positive code reviews including at least one code owner from each category referenced in the PR.
  • Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

  • Gatekeeper requested (daos-gatekeeper added as a reviewer).

@knard38 knard38 self-assigned this Jul 6, 2026
@github-actions

github-actions Bot commented Jul 6, 2026 •

Copy link
Copy Markdown

Ticket title is 'Add ASan/UBSan/TSan sanitizer checks to the GitHub Actions unit-test CI'
Status is 'In Review'
https://daosio.atlassian.net/browse/DAOS-18626

Base automatically changed from ckochhof/dev/master/daos-18826/patch-001 to master July 13, 2026 15:57
@daosbuild3

Copy link
Copy Markdown
Collaborator

@knard38
knard38 force-pushed the ckochhof/dev/master/daos-18626/patch-002 branch from de6bafa to 7b89b29 Compare July 22, 2026 08:52
@knard38
knard38 marked this pull request as ready for review July 23, 2026 09:55
@knard38
knard38 requested review from a team as code owners July 23, 2026 09:55
@knard38
knard38 marked this pull request as draft July 23, 2026 09:55
@knard38
knard38 force-pushed the ckochhof/dev/master/daos-18626/patch-002 branch from 7b89b29 to 6f22e4c Compare July 23, 2026 11:32
Add the build-system plumbing needed to compile DAOS with
UndefinedBehaviorSanitizer and ThreadSanitizer, in addition to the
already-supported AddressSanitizer:

- site_scons/site_tools/compiler_setup.py: link the UBSan/TSan runtime
  libraries (-lubsan/-ltsan) explicitly, including into CGO_LDFLAGS so
  that 'go build' picks them up when CGO-linking a sanitizer-compiled C
  archive.
- src/control/SConscript: skip control man-page generation under a
  SANITIZERS build (not needed, avoids extra build cost).
- src/gurt/misc.c: skip d_free()'s debug poisoning memset() under TSan,
  which otherwise turns a harmless zero-size-allocation address reuse
  into a false heap-use-after-free report.
- utils/scripts/install-el9.sh: install compiler-rt, libubsan, and
  libtsan alongside the existing libasan dependency.
- utils/docker/Dockerfile.el.9: thread a new SANITIZERS build-arg
  through to the scons invocation (empty/unset by default, so existing
  callers are unaffected).

This is purely additive: no existing (non-sanitizer) build is affected,
and SANITIZERS=address behavior is unchanged. Part of the DAOS-18626
ASan CI split (see ckochhof/dev/master/daos-18626/patch-001 for the
original, unsplit commit).

Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
@knard38
knard38 force-pushed the ckochhof/dev/master/daos-18626/patch-002 branch from 6f22e4c to ce9165c Compare August 10, 2026 09:14
@knard38
knard38 marked this pull request as ready for review August 12, 2026 08:33
Comment thread src/gurt/misc.c Resolved
Comment thread src/control/SConscript Resolved
Comment thread site_scons/site_tools/compiler_setup.py Resolved
kanard38 and others added 4 commits August 17, 2026 07:57
_f_get_alloc_size() was called unconditionally in d_free(), with its
result discarded via `(void)msize;` under TSan to silence
-Wunused-but-set-variable.

Scoping only the call site inside #if !defined(__SANITIZE_THREAD__)
leaves _f_get_alloc_size() with no caller left under TSan, tripping
-Werror=unused-function instead. Guard the function definition with
the same #if so it compiles out entirely under TSan rather than being
computed and discarded.

Verified with gcc -Wall -Wextra -Werror for plain, ASan, and TSan
builds.

Addresses PR #18613 review feedback from daltonbohning.

Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
Add a short code comment on why man-page generation is skipped when
building with SANITIZERS: it runs the just-built (sanitizer-instrumented)
binary as a build step, which can trip sanitizer false positives.

Suggested-by: daltonbohning
Link: #18613 (comment)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
@knard38
knard38 requested a review from daltonbohning August 18, 2026 06:30
@knard38
knard38 requested a review from liw August 27, 2026 08:53
@knard38

knard38 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

@grom72, @daltonbohning and @phender could you have a look at this PR please.
Thanks in advance.

@daosbuild3

Copy link
Copy Markdown
Collaborator

@kjacque kjacque left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically looks OK, I think it's worthwhile to update the confusing name in the build scripts though.

Comment thread site_scons/site_tools/compiler_setup.py Resolved
kanard38 and others added 2 commits September 1, 2026 06:50
Reviewer feedback (kjacque) pointed out that the SANITIZERS value
'undefined' for enabling UBSan is confusing -- easily misread as
value-not-set rather than as UBSan's own name (Undefined Behavior
Sanitizer). Rename the public SANITIZERS token to the clearer
'undefined_behavior', translating it back to the compiler's fixed
-fsanitize=undefined identifier at both points a -fsanitize=<name>
flag is constructed from a SANITIZERS token (_base_setup() and
_check_func(), which must stay symmetric: _check_func() strips the
exact flag _base_setup() added, so both need the same translation or
sanitizer flags stop being cleanly removed for CheckFunc() probes).

Also update site_scons/components/__init__.py, which independently
inspects SANITIZERS to pick Mercury's own CMAKE_BUILD_TYPE -- it must
recognize the new token too, or Mercury silently falls back to
RelWithDebInfo instead of Ubsan.

Link: #18613 (comment)
Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
@knard38
knard38 requested a review from kjacque September 1, 2026 09:05
kjacque
kjacque previously approved these changes Sep 2, 2026
PR #18613's Pylint check job failed with 2 wrong-spelling-in-comment
(C0401) findings for the word "fsanitize", both in the explanatory
comment added above SANITIZER_FLAG_NAMES in the previous commit
(rename SANITIZERS UBSan token to undefined_behavior).

Pylint's spell checker tokenizes "-fsanitize=" (stripping the leading
'-' and trailing '=') into the bare word "fsanitize", which isn't a
real English word and wasn't yet in utils/cq/words.dict. Fix by adding
it to the dictionary, in its correct alphabetical position.

Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>
daltonbohning
daltonbohning previously approved these changes Sep 4, 2026
kjacque
kjacque previously approved these changes Sep 10, 2026
@knard38
knard38 dismissed stale reviews from kjacque and daltonbohning via 68ae545 September 11, 2026 05:47
kjacque
kjacque previously approved these changes Sep 11, 2026
@knard38
knard38 requested a review from a team September 16, 2026 16:57
@knard38

knard38 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor Author

@daos-stack/daos-gatekeeper could you lend this PR with the following commit message:

  • title: DAOS-18626 build: Add ASan/UBSan/TSan compiler and Docker build support
  • body:
Add scons support for building DAOS with the AddressSanitizer,
UndefinedBehaviorSanitizer, and ThreadSanitizer compiler
instrumentation, and plumb SANITIZERS through to the Docker EL9
build image and its package list.

SANITIZERS now uses the token 'undefined_behavior' instead of the
compiler's own 'undefined' for the UBSan case, since the latter reads
as "value not set". A translation map converts it back to the real
-fsanitize= flag wherever a sanitizer flag is built or removed. The
UBSan/TSan runtime libraries are also linked explicitly (-lubsan,
-ltsan) and injected into CGO_LDFLAGS, since GCC only auto-links them
for pure-C targets and CGO would otherwise fail to link Go binaries
with "DSO missing from command line".

Two sanitizer-specific workarounds are included: d_free() now skips
its 0x42 memory-poisoning under TSan, since that otherwise turns
harmless zero-size-allocation address reuse into a false
heap-use-after-free; and the control man-page generation step, which
runs the just-built binary, is skipped when SANITIZERS is set to
avoid tripping sanitizer false positives during the build itself.

install-el9.sh and the EL9 Dockerfile gain the libasan/libubsan/
libtsan/compiler-rt packages and a SANITIZERS Docker build arg.

@daltonbohning daltonbohning added the release-3.0.0 Targeted for release 3.0.0 label Sep 16, 2026
@daltonbohning
daltonbohning merged commit 3e88c98 into master Sep 16, 2026
60 checks passed
@daltonbohning
daltonbohning deleted the ckochhof/dev/master/daos-18626/patch-002 branch September 16, 2026 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-3.0.0 Targeted for release 3.0.0

Development

Successfully merging this pull request may close these issues.

5 participants