Docs: creates markdown reference - #1565
Conversation
Wrap inline code examples in docstrings with ```python fences so the Markdown API reference renders them as highlighted code blocks instead of flat prose. Changes are docstring-only; no code behavior is affected. Co-Authored-By: Claude <svc-devxp-claude@slack-corp.com>
…down)
Replace the pdoc3 HTML generator with pydoc-markdown so the API reference
is emitted as Markdown for docs.slack.dev/Docusaurus.
- Rewrite generate_api_docs.sh to drive the new Markdown pipeline.
- Add generate_api_docs.py, which:
- inlines re-exported classes/functions so adapter pages show their
handler inline (matching pdoc3's behavior);
- adds OrderedGoogleProcessor to keep fenced code blocks in their
original position (the stock GoogleProcessor relocates a code block
that precedes a section keyword to after the prose);
- replaces pydoc-markdown's escape_except_blockquotes, which corrupts
docstrings with >10 code spans by duplicating a code block into later
spans (BLOCKQUOTE_TOKEN prefix collision).
- Regenerate docs/reference as Markdown (removes the old HTML tree).
- Point the sidebar "Reference" link at the new Markdown path.
Co-Authored-By: Claude <svc-devxp-claude@slack-corp.com>
Rename each generated package __init__.md to index.md and rewrite the generated sidebar.json edges to match. The docusaurus renderer emits a package's docs as <pkg>/__init__.md, whose route is .../<pkg>/__init__ -- nothing resolves at the bare .../<pkg>/ URL that the sidebar's Reference link (.../reference/slack_bolt/) targets. Docusaurus serves index.md at the folder URL, so this makes that link resolve instead of 404. Co-Authored-By: Claude <svc-devxp-claude@slack-corp.com>
The App.start() docstring had an indented (unfenced) code example whose '#' comment lines rendered as Markdown H1 headers in the Markdown output. Wrap it in a ```python fence. This propagates to all 24 pages that inline App via re-export. Also point the generator at docs/english/reference (the reference tree's location) instead of docs/reference. Co-Authored-By: Claude <svc-devxp-claude@slack-corp.com>
The docs site imports docs/english/_sidebar.json and filters it; it does not read the generated reference/sidebar.json. Replace the external "Reference" link with the generated category tree, prefixing doc IDs with tools/bolt-python/ so they resolve against the docs root. The generator now does this automatically (_sync_reference_sidebar) so the sidebar stays in sync on every regeneration. Co-Authored-By: Claude <svc-devxp-claude@slack-corp.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1565 +/- ##
=======================================
Coverage 91.54% 91.54%
=======================================
Files 228 228
Lines 7285 7285
=======================================
Hits 6669 6669
Misses 616 616 ☔ View full report in Codecov by Harness. |
WilliamBergamin
left a comment
WilliamBergamin
left a comment
There was a problem hiding this comment.
Thanks for working on this 💯
Left some comments/questions before we can move forward with this 🙏
| @@ -0,0 +1,756 @@ | |||
| #!/usr/bin/env python | |||
There was a problem hiding this comment.
This is a lot of custom code 😅 any chance we can rely on defaults or configs instead of having this?
|
@WilliamBergamin Using a new library that's a bit more consistent. Still a lot of custom code, but that's just the nature of the beast given our unique docusaurus setup - it's mostly on that docusaurus side, not the python repo side |
WilliamBergamin
left a comment
WilliamBergamin
left a comment
There was a problem hiding this comment.
I like like the direction here. Moving the reference to Markdown is a clear win over the pdoc3 HTML blob:
- the pages are agent-readable
- they can be tracked in GA
- they slot into Docusaurus natively.
griffe is also the right parser choice (it resolves our __all__ re-exports and parses the Google-style docstrings correctly).
My main concern is scripts/generate_api_docs.py as a long-term maintenance surface: ~640 lines of hand-rolled code is a lot to keep working as slack_bolt, griffe, and Docusaurus all evolve. I dug into how the other tools ship their reference and I think we can remove a big chunk of the hand rolled code.
node-slack-sdk is the closest working model. It ships a committed Markdown reference that docs.slack.dev consumes, and it does two things we're currently doing the hard way:
- No hand-built
sidebar.json. It uses a Docusaurusautogeneratedsidebar (dirName+ agenerated-indexlink);deno-slack-sdkandslack-clido the same. Worth noting: the docs site imports exactly one_sidebar.jsonper tool today and has no mechanism to import a secondreference/sidebar.json, so the artifact we generate can't actually be wired in as-is. Switching to anautogeneratedReferencecategory in our existingdocs/english/_sidebar.jsonremoves ~90 lines and the whole "how does the site pick this up" question. - Signatures in fenced code blocks. MDX leaves
<and{literal inside code fences, so the escaping only really needs to cover prose docstring text, not signatures. That shrinks the escaping/hazard surface a lot.
What I'd suggest (keep griffe, cut the rest):
- Drop the
sidebar.jsongeneration; add anautogeneratedReferencecategory todocs/english/_sidebar.json(node/deno/slack-cli pattern). - Move the imperative signature/section rendering into a small set of Jinja2 templates (the mkdocstrings/quartodoc approach). Same output, roughly half the code, and future tweaks become template edits.
- Keep a minimal prose-level MDX escaper + the hazard gate, and wire the hazard gate into CI.
- Pin
griffein a requirements file (it's currentlypip install -U griffe, so output can shift under us), and add a CI job that regenerates and fails ongit diffdrift, since the generated tree is committed and nothing guards it right now.
Happy to pair on the template refactor if useful.
2 participants
Summary
This PR replaces the html reference with a md-based on. Tighter integration with docusaurus, and allows easier agent access as the md files are accessible. Also will now be trackable in Google Analaytics
Testing
Category
slack_bolt.Appand/or its core componentsslack_bolt.async_app.AsyncAppand/or its core componentsslack_bolt.adapter/docsRequirements
Please read the Contributing guidelines and Code of Conduct before creating this issue or pull request. By submitting, you are agreeing to those rules.
./scripts/install_all_and_run_tests.shafter making the changes.