Skip to content

Add optional data dictionaries and refresh Connect pin previews - #364

Open
mconflitti-pbc wants to merge 4 commits into
mainfrom
mconflitti/data-dict
Open

mconflitti-pbc wants to merge 4 commits into
mainfrom
mconflitti/data-dict

Conversation

@mconflitti-pbc

@mconflitti-pbc mconflitti-pbc commented Nov 17, 2026 •

Copy link
Copy Markdown

Pins can attach a portable data dictionary to a pin and include a browsable dictionary in its Posit Connect preview. The publisher runs the optional data-dict CLI before uploading the bundle; Connect serves the resulting HTML alongside the existing data and metadata.

  • Add data_dict=True to draft from a DataFrame in pin_write() or Parquet files in pin_upload(). Supply an authored YAML file path, or YAML text in metadata["data_dict"], to attach an existing dictionary. Stored dictionaries omit table source paths.
  • Add the optional pins[data-dict] extra, with an executable on PATH as a fallback. Handle pip wheels with a missing execute bit and require supported CLI version 0.0.3. Ordinary pins do not require the CLI; attaching authored YAML to a folder board works without it.
  • Refresh the Connect preview with Overview and Metadata tabs, usage code, downloads, and an optional Data dictionary tab. Metadata is displayed directly. The preview and dictionary share Connect's theme and default to light when no preference is available.
  • Embed column profiles when a pin contains one Parquet file and one dictionary table. Other dictionaries still render their definitions without an inferred source mapping.
  • Document force_identical_write=True and explain skipped dictionary-only edits when the data hash is unchanged. Reject reserved preview filenames, report file/CLI errors through PinsError, and clean up failed renders.
  • Defer CLI drafting and rendering until after the identical-data check. Protect generated dictionary HTML with a script hash policy and strip unsafe link schemes.
  • Add synthetic-data examples and a dedicated CI job that installs the optional extra and requires the draft/render integration tests to run.
board.pin_write(df, "username/data", type="parquet", data_dict=True)
board.pin_write(
    df, "username/data", type="parquet",
    data_dict="data-dict.yaml", force_identical_write=True,
)

Validation: 47 dictionary/preview tests passed with the CLI required; 42 existing folder-board tests passed (2 unrelated skips). Ruff lint and format checks passed. A clean macOS pip install reproduced the wheel permission issue and the CLI fix succeeded.

Redeployed ordinary and combined Parquet previews from both local branches to Connect. Downloaded the active bundles, verified they match the builds, and read all four pins back in both languages. Browser checks confirmed light theme, direct metadata display, interactive column profiles, and blocked executable dictionary links. Python CI exercises the optional extra on Linux, macOS, and Windows; R CI exports the installed CLI path into R CMD check. Full CI runs on this PR; CRAN readiness has not been established.

This branch was successfully deployed

1 active deployment
pr-364 — f6430b6e Deployed Nov 17, 2026 by github-actions[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant