generated API reference with DocFX #337

[FreeAndNil]

• docfx config in src/site/docfx, output target/site/api
• build-api-docs.ps1, warnings as errors, in build-release.ps1 and CI
• header link in antora playbook
• old versions regenerated by scripts/archived/build-old-api-docs.ps1
• shared helpers in scripts/Common.psm1

Try on staging:

• current layout: https://logging.staged.apache.org/log4net/log4net-3.5.0/api/index.html
• Ext.Mail, base class links into log4net: https://logging.staged.apache.org/log4net/log4net-3.5.0/api/log4net.Ext.Mail/log4net.Ext.Mail.Appender.SmtpAppender.html
• header link on an Antora page: https://logging.staged.apache.org/log4net/log4net-3.4.0/manual/configuration/appenders/fileappender.html
• menu entry on a Maven page: https://logging.staged.apache.org/log4net/log4net-3.0.4/release/faq.html
• old Sandcastle link, redirected: https://logging.staged.apache.org/log4net/log4net-2.0.8/release/sdk/html/M_log4net_ILog_DebugFormat_2.htm
• old NDoc link, redirected: https://logging.staged.apache.org/log4net/log4net-1.2.11/release/sdk/log4net.ILog.Debug_overload_1.html
• a type that only 2.0.8 has: https://logging.staged.apache.org/log4net/log4net-2.0.8/release/sdk/log4net/log4net.Appender.RemotingAppender.html

[FreeAndNil]

Implementation plan for #337, kept here for the record. Written up to describe the change as it stands on the branch.

Plan

API reference (DocFX) for the log4net site

Context

Until 2.0.8 every release shipped release/sdk/, a Sandcastle Help File Builder rendering of the
XML doc comments, and 1.2.11 to 1.2.13 an NDoc one. The SHFB project was deleted in 2024
(20cf9dca) and nothing replaced it, so 2.0.9 through 3.5.0 have no API reference. The XML doc
files are still produced on every build (GenerateDocumentationFile in src/Directory.Build.props)
and CS1591 is enforced for the shipping libraries, so the input is complete.

Goal: generate the reference with DocFX into target/site/api/, published next to the Antora
output as logging.apache.org/log4net/api/ and as log4net-<version>/api/ per release, for both
log4net and log4net.Ext.Mail. The step runs from build-release.ps1 and in CI with warnings as
errors. The versions already published in logging-log4net-site get the same reference from their
NuGet packages, replacing the Sandcastle and NDoc pages in place, with the old links kept valid by
rewrite rules.

Decisions taken with the user: DocFX; output path api/; both assemblies; release script plus CI;
replace the old SDK folders in place; generator for the old versions kept under scripts/archived/;
packages from nuget.org rather than the Apache archive, which lacks 1.2.11 to 1.2.14 and 2.0.11 and
ships three zip layouts.

Constraints that shape the design

• log4net targets net462;netstandard2.0; the net462 surface is the superset (EventLogAppender,
  AspNetTraceAppender, the AspNet*PatternConverters and WindowsSecurityContext are
  NET462_OR_GREATER only). log4net.Ext.Mail targets netstandard2.0 only. One DocFX metadata
  entry carries one TargetFramework, so there are two entries with dests log4net and
  log4net.Ext.Mail inside the docfx folder, joined by a hand-written toc.yml. Published URLs are
  api/log4net/<type>.html and api/log4net.Ext.Mail/<type>.html.
• Roslyn matches a project reference to its metadata only when the referenced project has a
  reference-assembly output path, which netstandard2.0 projects lack by default. Without
  ProduceReferenceAssembly=true on the Ext.Mail entry, docfx warns "Found project reference
  without a matching metadata reference", and the warning cannot be suppressed. The property is a
  design-time global for the docfx build only.
• DocFX validates every relative link, and the manual lies outside its source folder. A
  manual.md with redirect_url: ../index.html is a valid link target that forwards to the
  manual, at the root and under a versioned folder alike.
• The modern template has no size rule for the logo and renders the brand as a flex row, so a
  custom template layer (template/public/main.css) sets the logo height and a right margin.
• DocFX has no year variable, so the footer, the Antora footer with its license and privacy policy
  links, is passed by the script with -m _appFooter=... and the year computed. Nothing is
  hardcoded in docfx.json.
• rat.excludesFile already excludes *.json, *.yml, *.md, *.ps1 and src/site/**;
  *.psm1 is added for the shared module.
• build-release.ps1 removes target/ before ./mvnw site, so docfx runs after the site. Maven
  itself does not wipe target/.
• PowerShell variable names are case-insensitive, so a loop variable must not shadow a script
  variable by case ($work versus $Work). pwsh -File hands a shell's 2.0.8,3.4.0 over as one
  string, so a version list is split on commas. A module takes preference variables from the
  global scope, so Common.psm1 sets $ErrorActionPreference itself.
• DocFX reads XML doc comments differently from Sandcastle, and three of the old conventions break
  it. Text on the same line as <example> makes it indent the <pre> by four extra spaces, which
  markdown renders as literal text; the text goes on its own line. Raw XML inside
  <code escaped="true"> is parsed as child elements and dropped, leaving an empty block; the XML
  goes into a CDATA section. The code language comes from a language attribute with the
  highlight.js names (csharp, xml, sql, vbnet); lang is ignored and everything falls back
  to C#. The examples of the context stack types are reindented alike. The packaged XML of the
  published versions keeps the old conventions and is left as it is; only the 3.5.0 folder is
  replaced by a build from the current source with -Version 3.5.0, which has the same public
  surface.
• The old Sandcastle html/ folder is gone after the rebuild, so Apache treats a page name under it
  as PATH_INFO and mod_rewrite re-appends it after every substitution; the DPI flag discards it.
  The final rewrite is an external 301, so the browser lands in the real folder and the relative
  links of the DocFX page resolve.

Files

Current version (this repo)

1. .config/dotnet-tools.json: pins docfx 2.81.0 (isRoot: true, rollForward: false). Not in
   src/Directory.Build.props, which holds NuGet floors for PackageReference; nothing bumps the
   pin automatically.
2. src/site/docfx/docfx.json: two metadata entries (log4net/log4net.csproj with
   TargetFramework: net462, dest log4net; log4net.Ext.Mail/log4net.Ext.Mail.csproj with
   TargetFramework: netstandard2.0 and ProduceReferenceAssembly: true, dest log4net.Ext.Mail;
   src: "../..", namespaceLayout: nested, memberLayout: samePage, disableGitFeatures).
   build: content index.md, manual.md, toc.yml, log4net/**/*.yml,
   log4net.Ext.Mail/**/*.yml; resource logo.svg; dest: ../../../target/site/api; template
   default, modern, template; globalMetadata _appTitle, _appName, _appLogoPath and
   _appFaviconPath (logo.svg, no site version ships a favicon.ico), _enableSearch,
   _disableContribution.
3. src/site/docfx/toc.yml (ASF header): log4net/, log4net.Ext.Mail/, Manual -> manual.md.
4. src/site/docfx/index.md: heading "API Reference", xref links to log4net and
   log4net.Ext.Mail.Appender (the Ext.Mail root namespace page), link to manual.md.
5. src/site/docfx/manual.md: front matter redirect_url: ../index.html only.
6. src/site/docfx/logo.svg: copy of the Antora logo, so the CI build is self-contained.
7. src/site/docfx/template/public/main.css (ASF header): #logo { height: 2.5rem; width: auto; margin-right: 0.75rem; }.
8. scripts/Common.psm1: Invoke-InDirectory, Remove-Directory, Get-ApiDocsFooter, with
   Export-ModuleMember and its own $ErrorActionPreference = 'Stop'. Imported by
   build-release.ps1, build-api-docs.ps1 and archived/build-old-api-docs.ps1; the release
   script loses its private copies.
9. scripts/build-api-docs.ps1 [-Version]: removes the generated metadata folders and
   target/site/api, dotnet tool restore in the repo root, dotnet docfx <config> --warningsAsErrors -m "_appTitle=Apache log4net <version> API Reference" -m "_appFooter=<footer>", throws if no index.html came out. The version defaults to the
   VersionPrefix of src/Directory.Build.props; build-release.ps1 passes its own.
10. scripts/build-api-docs.Tests.ps1: tool restore failure stops before docfx; docfx failure
    exits non-zero with calls dotnet tool, dotnet docfx; --warningsAsErrors passed once and
    index.html present; missing output throws; a planted stale file is removed; the footer carries
    the current year and the privacy policy link; the title names the props version or the one
    given.
11. scripts/FakeCommands.TestHelper.ps1: FAKE_FAIL matches a command or command plus first
    argument (dotnet docfx); the dotnet fake dispatches on $args[0] (tool writes nothing,
    docfx writes index.html at the -o base or at target/site/api, unless FAKE_NO_OUTPUT);
    a curl fake writes a real zip with lib/net462 and lib/netstandard2.0 assemblies; the
    scratch tree gets build-api-docs.ps1, Common.psm1, the docfx logo and template, a props file
    with VersionPrefix 9.9.9, and keeps a script's subfolder; Invoke-InScratchTree takes -NoOutput and -Arguments.
12. scripts/build-release.ps1: imports the module; after ./mvnw site runs
    & $PSScriptRoot/build-api-docs.ps1 -Version $Version. Tests: 'dotnet tool', 'dotnet docfx'
    appended to the full-run call list; the release version in the title; "asks for no tag when
    the API reference build fails".
13. .github/workflows/build.yaml: step "API reference" after Test, Ubuntu only, shell: pwsh,
    run: ./scripts/build-api-docs.ps1.
14. .gitignore: /src/site/docfx/log4net/ and /src/site/docfx/log4net.Ext.Mail/.
15. rat.excludesFile: **/*.psm1.
16. antora-playbook.yaml, header-content.hbs partial: <a class="navbar-item" href="{{{siteRootPath}}}/api/index.html">API Reference</a> before the GitHub link. The UI
    relativizes it per page (./api/index.html at the root, ../../../api/index.html three levels
    deep). nav.adoc unchanged. One sentence each in manual/index.adoc
    (link:../api/index.html[API reference]) and the [#site] section of development.adoc.
17. doc/BUILDING.md: subsection "API reference" under "Updating the site". doc/RELEASING.md:
    step 2 notes the API reference build and the network for the tool restore, step 4 says
    "including api/". AGENTS.md: a paragraph on the reference in "Documentation site", and in
    "Release scripts" the rule to grep scripts/ for a helper's name and idiom before writing one,
    with shared functions in Common.psm1; the curl shim in the helper list. Nothing about the
    archived script in either.
18. Doc comments in src/log4net: example text on its own line in ten files, the twelve
    escaped="true" XML blocks in CDATA in seven files, lang renamed to language with
    highlight.js names in all 42 blocks, the context stack examples reindented.
19. src/changelog/3.5.1/337-api-reference.xml: type added, issue 337, prose: published at
    api/, built with DocFX, checked in CI with warnings as errors, the published versions
    regenerated the same way with the old links kept valid (implemented by @FreeAndNil).

Published versions (site repo), one-off

20. scripts/archived/build-old-api-docs.ps1 -SiteRoot <checkout> [-Version a,b]: per
    log4net-<version> folder, oldest first: download log4net.<v>.nupkg from the nuget.org flat
    container with curl --fail as a .zip (Expand-Archive wants the extension), 1.2.12 to 1.2.15
    mapped to 2.0.1, 2.0.3, 2.0.4, 2.0.5; Ext.Mail from 3.4.0; lib folder preferred net462,
    net45, net45-full, net40-full (Ext.Mail netstandard2.0). Generates per version a docfx
    folder under target/old-api/<v>/ with assembly input, the shared logo and template, its own
    toc.yml, index.md and manual.md, and runs docfx without warnings as errors with
    -o <dest>, _appTitle carrying the version and the footer. A folder with release/ gets
    release/sdk/ (manual redirect ../../index.html) and, where its Maven menu lacks it, an
    "SDK Reference" entry after "FAQ" whose href is derived from the FAQ link, the FAQ entry
    carrying class="none active" on the FAQ page itself; an Antora folder
    gets api/ (manual redirect ../index.html) and the "API Reference" header link before the
    GitHub link, with the relative path per page. Both insertions are idempotent and skip empty
    pages. Each version in its own work folder.
21. scripts/archived/build-old-api-docs.Tests.ps1: fixture with 2.0.8 (release layout, old
    Sandcastle pages), 3.0.4 (release layout, Maven menu without SDK entry, its FAQ page with the
    active class) and 3.4.0 (Antora header). Cases: no site folder, download failure, Sandcastle
    pages replaced, menu entry added once with the right relative path and on the active FAQ page
    (counting the link text, since the title attribute repeats it), header link added once, comma
    separated version list,
    all folders by default in separate work folders, 1.2.15 downloaded as 2.0.5.
22. .htaccess in the site repo: Sandcastle rules ([MPFE]/Overload_ member pages lose an
    overload index or __ctor suffix and then their last segment, member lists AllMembers_ etc.
    go to the type, T_/N_ names get their underscores turned into dots and are redirected
    R=301 into release/sdk/log4net/, all with DPI); NDoc rules (a name that exists as a
    DocFX type or namespace page under log4net/ is redirected to it, checked with a
    RewriteCond ... -f derived from %{REQUEST_FILENAME}; otherwise Members, Methods,
    Properties, Constructor\d*, _overloads, _overload_\d+ suffixes or the last segment are
    stripped with [N] until one exists). The old AllMembers_ rule and the unversioned
    catch-all to the current version stay as they were in spirit; the catch-all remains.
    Known gaps: explicit interface implementation pages, identifiers with underscores.

Sequencing

1. Items 1 to 7, 14, 15; run docfx by hand with warnings as errors, from a clean state without a
   Release build.
2. Items 8 to 12, run Pester.
3. Item 13.
4. Items 16 to 19; the doc comment fixes are their own commit.
5. Items 20 and 21; run the archived script against the site checkout, every version.
6. Item 22, tested in a podman httpd:2.4 container with mod_rewrite and AllowOverride All
   against the checkout, then on staging. The harness: a cases.txt of old URL and expected
   title pairs, checked with curl -L for status 200, the title and an effective URL inside a real
   folder.
7. Commits and pushes are the user's.

Verification

• dotnet build src/log4net.sln > out.txt 2>&1: 0 warnings.
• pwsh ./scripts/build-api-docs.ps1: 0 warnings, target/site/api/index.html,
  log4net/log4net.Appender.FileAppender.html, EventLogAppender and AspNetTraceAppender pages
  (proves net462), log4net.Ext.Mail.Appender.* pages linking into the log4net pages, BCL links
  to learn.microsoft.com, logo at navbar height with a gap before the title, footer with year and
  privacy link, manual.html a meta refresh to ../index.html. Serve over HTTP to see the navbar
  and search: the modern template fetches toc.json, which Chrome blocks for file://.
• Run the script twice; the second run removes a planted stale file.
• Invoke-Pester ./scripts: all suites green, the archived one included (discovered recursively).
• ./mvnw site then the script; the header link resolves at every page depth and from a copy
  under a log4net-<version>/ folder.
• No page of the current build contains <pre><code or an empty
  <pre><code class="lang-csharp"></code></pre>, and the code blocks carry lang-xml, lang-sql
  and lang-vbnet where the source says so.
• Site checkout after the archived run: every folder has release/sdk/log4net/ or
  api/log4net/ with 225 to 235 type pages, no html/ folder left, Ext.Mail pages from 3.4.0,
  header or menu link on every page that lacked one.
• The rewrite harness passes all cases in the container and on staging, with the 1.2.x NDoc cases
  once those folders are rebuilt.

[fluffynuts]

nice one (: