From b4b9fb64893a6fbdab281abc1dfcd3565152a583 Mon Sep 17 00:00:00 2001 From: MendixMau Date: Sun, 20 Sep 2026 19:09:18 +0000 Subject: [PATCH 1/2] marketplace: anchor the project dir before the bundled-file containment check MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `mxcli marketplace install ... -p app.mpr` — a project named by a RELATIVE path — failed with `package entry "manifest.json" would write outside the project`. filepath.Dir("app.mpr") is ".", filepath.Join drops the dot, and the joined path "manifest.json" was compared against the prefix "./", so every legitimate entry tripped the zip-slip guard. The module had already been transplanted into the model by then, so the command exited 1 over a half-finished install: SHOW MODULES lists it, mx check is clean, no bundled file on disk. Make projectDir absolute before the loop. The guard itself is unchanged and its traversal tests still pass. The new test fails on the unpatched function with the exact reported message (prove-by-revert). Finding recorded in .claude/skills/fix-issue/findings/cmd-mxcli.jsonl. Found by the first field run of `marketplace install --file`; the same code runs for an online install. Co-Authored-By: Claude Fable 5.1 --- .../skills/fix-issue/findings/cmd-mxcli.jsonl | 1 + CHANGELOG.md | 2 + .../marketplace/install_relative_dir_test.go | 53 +++++++++++++++++++ cmd/mxcli/marketplace/update.go | 7 +++ 4 files changed, 63 insertions(+) create mode 100644 cmd/mxcli/marketplace/install_relative_dir_test.go diff --git a/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl b/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl index d78263672..5859918bf 100644 --- a/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl +++ b/.claude/skills/fix-issue/findings/cmd-mxcli.jsonl @@ -117,3 +117,4 @@ {"area": "cmd-mxcli", "date": "2026-09-17", "symptom": "`mxcli new --version 10.24.25` (and `mxcli setup mxbuild --version 10.24.25`) dies with `HTTP 404 from https://cdn.mendix.com/runtime/mxbuild-10.24.25.tar.gz`. Every 9.x and 10.x version probed 404s while 11.6.0/11.12.1/11.13.0 return 200 from the same host and path, which reads as 'Mendix 10 is no longer on the CDN'.", "cause": "Mendix 9 and 10 publish FOUR-part artifact names carrying a build number the release notes never mention: the release called 10.24.25 is `mxbuild-10.24.25.122571.tar.gz`. Mendix 11 publishes three parts. `MxBuildCDNURL` interpolates whatever string it is handed and nothing resolved a partial version, so a hand-typed 10.x version named no artifact at all. Project-driven paths were never affected — the MPR's `_ProductVersion` already carries all four parts (`10.24.25.122571`) and `parseVersion` takes the first three for major/minor/patch while the full string goes to the URL.", "file": "`cmd/mxcli/docker/version_resolve.go` (ResolveCDNVersion, highestBuild, CDNReleasesFor); wired at the two entry points where a user types a version, `cmd/mxcli/cmd_new.go` and `cmd/mxcli/setup.go`. Tests `cmd/mxcli/docker/version_resolve_test.go`.", "insight": "**A uniform 404 across a whole major version is evidence about the NAME, not about availability.** The conclusion drawn from it — 'Mendix 10 cannot be downloaded here' — blocked a verification for an entire session, and the fix was one listing call: the CDN is an S3 bucket that answers ListObjectsV2 (`?list-type=2&prefix=runtime/mxbuild-10.24.`), so what exists is enumerable rather than guessable. When a probe fails identically for every input in a class, question the query before concluding the class is empty. Three traps in the resolution itself, each a test: the `.sha256` sidecar beside every archive must not be picked as an artifact; the prefix needs its trailing dot or `10.24.2` swallows `10.24.20`..`10.24.26`; and build numbers are not zero-padded, so a text sort puts 99999 above 122571 and 10.24.9 above 10.24.26. Resolve at the entry point and thread the RESOLVED string onward — `mxcli new` checks the created project's stamp against the requested version, and `mx create-project` stamps four parts, so resolving late would fail that postcondition.", "refs": ["#1121"]} {"area": "cmd/mxcli", "date": "2026-09-18", "symptom": "mendixlabs/mxcli#1025: `mxcli syntax` advertises `mxcli syntax workflow user-task targeting` in its own help and answers `Unknown topic: workflow user-task targeting`. Same for `workflow user-task` and `workflow parallel-split`, all of which `mxcli syntax workflow` lists as sub-topics; `--json` was the only route that reached them.", "cause": "The CLI built its path with `strings.Join(args, \".\")` and never split an argument, so a topic handed over as ONE string — a quoted copy-paste, a tool wrapper, `sh -c` — became the path `workflow user-task targeting`, which matches nothing. The REPL's `help` had resolved multi-word topics since it was written (`resolveHelpPath`, greedy hyphen-joining): one question, two answers, and the CLI held the weaker copy. The #955 segment-match fallback could not save it either — it passed the DOTTED path to `BySegmentMatch`, and no segment contains a '.', so that fallback was silently dead for every multi-word query.", "file": "cmd/mxcli/syntax/topic.go (new: Lookup, topicWords, resolvePath), cmd/mxcli/help.go, mdl/executor/cmd_misc.go (resolveHelpPath deleted), mdl/grammar/domains/MDLSettings.g4 (helpStatement, helpTopicWord), mdl/visitor/visitor_query.go (ExitHelpStatement); tests cmd/mxcli/cmd_syntax_test.go, cmd/mxcli/syntax/topic_test.go, mdl/executor/cmd_misc_test.go, mdl/visitor/visitor_help_topic_test.go; example mdl-examples/bug-tests/syntax-1025-topic-drilldown.mdl", "insight": "**The spaces in the reported error message were the whole diagnosis, and reading them as a paraphrase cost an hour.** The command prints the path it built, and the CLI joins on '.', so `Unknown topic: workflow user-task targeting` cannot come from the command as documented — it can only come from the topic arriving as a single argument. Every line of the report follows from that and nothing else does: `syntax workflow` works (one word), `--json` works (the flag is not part of the topic), the three multi-word forms fail. Take a quoted error message literally, character for character, before assuming the reporter retyped it. **The reported version is downloadable and settles it in one run**: `mxcli setup mxcli`'s own URL shape (`releases/download//mxcli-linux-amd64`, NOT the goreleaser `_Linux_x86_64.tar.gz` that 404s) fetched v0.20.0, where the unquoted command works and the quoted one reproduces the message verbatim — so 'fixed since' and 'never broken' were both wrong. **The guard that matters is not the three cases from the report** but `TestEveryRegisteredPathIsReachableBySpelling`: every registered path, tried dotted, as separate arguments, and as one string. The registry prints dotted paths and then tells the reader to drill down with words, so a spelling that does not resolve is the command contradicting its own output; a per-case test would have passed the day someone added a topic with a new shape. Control: stub the whitespace split in `topicWords` and it fails with the reported path, spaces and all. **The grammar half has a trap the CLI half does not, and only the EXISTING suite caught it.** `helpStatement: IDENTIFIER (identifierOrKeyword)*` is the grammar's catch-all — a statement that is just an identifier and some words — so whatever it can swallow, it swallows from the statement that should have had it. Widening it to `(DOT? helpTopicWord)*` to take `help workflow.user-task` made `Sec.ApiUser` a complete statement of its own, and `create module role Sec.ApiUser` then parsed, WITH NO PARSE ERROR, as CREATE MODULE (named \"role\") followed by a help topic — two statements, wrong types, six unrelated security tests red. `(helpTopicWord (DOT? helpTopicWord)*)?` — a topic word before any dot — leaves `.ApiUser` unconsumable and restores the old disambiguation. Bisect a grammar regression by SHAPE, not by reading the ATN: adding the unused rule alone was clean, the hyphen alone was clean, the leading optional DOT was the whole of it, and three regenerations said so in about a minute. **When widening a permissive rule, the test to add is not for the new spelling but for what the rule must still NOT swallow** (TestHelpRuleDoesNotSwallowATrailingQualifiedName).", "refs": ["mendixlabs/mxcli#1025", "#955"]} {"area": "cmd/mxcli", "date": "2026-09-18", "symptom": "`mxcli report` scores a project against rules the team disabled in `lint-config.yaml`. `mxcli lint` honours the config, the report's SCORE does not move, so the score cannot be calibrated at all. Reported at 66/100 against a 99/100 blank-app baseline, where 61 of 86 findings were two deliberately-accepted rules", "cause": "`cmd_report.go` never called `linter.FindConfigFile`/`LoadConfig` — it went straight from `linter.New` to `BuildReport`. Separately it carried its own INLINE copy of the built-in rule list, one rule behind `builtinLintRules()` (missing MDL-FLOW01), so the two commands scored one project against two rule sets. One root cause: report re-implemented lint's setup instead of sharing it", "file": "`cmd/mxcli/cmd_report.go`, `cmd/mxcli/cmd_lint.go`, new `cmd/mxcli/lint_setup.go` (`projectLintRules`, `applyLintConfig`), `mdl/linter/linter.go` (`RuleEnabled`)", "insight": "Same class as #904 in the opposite direction: there a silently reduced rule set made the score falsely HIGH, here an unread config makes it falsely LOW — and both are invisible because a score carries no provenance. **A value test cannot guard the inline copy**: both commands build rules inside a cobra RunE, so nothing a unit test can call notices a second list being re-added. The guard is therefore structural — grep `cmd_report.go` for `lint.AddRule(rules.New` — with a POSITIVE CONTROL first (assert `builtinLintRules` still constructs rules) so it cannot pass vacuously, the same shape as `scripts/check-tunnel-deps.sh`. Take the LintContext out of `applyLintConfig`'s signature: `NewLintContext(nil, nil)` panics, and a nil-guard added only to make a test compile is how a helper acquires behaviour nothing needs", "refs": ["#525", "#904"]} +{"area": "cmd/mxcli/marketplace", "date": "2026-09-20", "symptom": "`mxcli marketplace install ... -p app.mpr` (project named by a RELATIVE path) fails with `install the package's bundled files: package entry \"manifest.json\" would write outside the project` \u2014 after the module has already been transplanted into the model. `SHOW MODULES` lists the module, `mx check` is clean, but no bundled file (themesource/, widgets/) landed and the command exited 1. Absolute `-p` paths work.", "cause": "`InstallPackageFiles` builds `dst := filepath.Join(projectDir, clean)` and then checks `strings.HasPrefix(filepath.Clean(dst), filepath.Clean(projectDir)+os.PathSeparator)`. With projectDir == \".\" (from `filepath.Dir(\"app.mpr\")`), Join drops the dot, so dst is `manifest.json` and the prefix is `./` \u2014 every legitimate entry fails the zip-slip guard. The guard was checking the joined path (the shape CodeQL recognises) but never anchored the project directory first.", "file": "`cmd/mxcli/marketplace/update.go` (`InstallPackageFiles`: `filepath.Abs(projectDir)` before the loop), test `cmd/mxcli/marketplace/install_relative_dir_test.go`", "insight": "A containment guard has two inputs and both must be canonical \u2014 the entry AND the root. The traversal tests only ever passed an absolute t.TempDir(), so the root was canonical by accident and the relative case had no coverage; the first real CLI invocation with `-p app.mpr` hit it. Worse, the transplant runs BEFORE the file step, so the failure lands on a half-installed module: the model has it, the disk does not, and the exit code says failure. Order the steps so the cheap, reversible file copy can be validated before the model write, or at least say in the error that the model was already changed. Prove-by-revert done: the new test fails on the unpatched function with the exact reported message.", "refs": []} diff --git a/CHANGELOG.md b/CHANGELOG.md index cb52184cc..36f6bf53a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Fixed +- **`marketplace install` refused a package's own bundled files when the project was named by a relative path** (`-p app.mpr`): `InstallPackageFiles` joined `filepath.Dir("app.mpr") == "."` with the entry, which drops the dot, then checked the result against the prefix `./` — so `manifest.json` was rejected as "would write outside the project". The module had already been transplanted into the model by then, so the command exited 1 over a half-finished install (model updated, no `themesource/` or `widgets/` on disk). The project directory is now made absolute before the containment check; the zip-slip guard is unchanged and its tests still pass. Found by the first field run of `--file`; the same code runs for an online install. Test added, shown to fail on the unpatched function with the exact message. + - **`mxcli run --local` hung forever on Windows at `Starting mxbuild --serve...`** — two POSIX assumptions in `cmd/mxcli/docker` combined. `ServeServer.alive()` (and `LocalRuntime.alive()`) asked `os.Process.Signal(syscall.Signal(0))`, which Go implements as `EWINDOWS` ("not supported by windows") for every signal but `Kill` — so a live mxbuild read as dead and `waitReady()` aborted the boot the instant it launched it. And `Stop()` waited on `cmd.Wait()` after a `killProcessGroup` that on Windows only called `p.Kill()`: mxbuild.exe is a wrapper that launches a Deno web-ext worker (`modeler/tools/deno/win-x64/deno.exe`) which inherits the stdout/stderr pipe `exec.Cmd` hands out, so the orphan kept the pipe open and `Wait()` never saw EOF — no error, no exit, just a hang. Windows now has a real liveness check (`OpenProcess(SYNCHRONIZE)` + `WaitForSingleObject`) and a tree kill (`taskkill /F /T`) behind `signalProcessGroup`/`killProcessGroup`; the POSIX implementations are unchanged. The Windows-only tests in `procgroup_windows_test.go` pin both halves — the tree-kill test fails on the pre-fix code, verified as a control — and a `windows-latest` CI job runs them (asserting they actually executed) so neither can come back. - **`raise error;` on a microflow's main flow passed check and exec, then failed the build** (mendixlabs/mxcli#1030) — with `[error] [CE0710] "The main flow cannot join an error flow or end in an error event."`, one per microflow. Mendix's error event *re-raises the error being handled*, so it is legal only where an error is in scope: inside an `on error { … }` handler. Studio Pro will not draw the connection from the normal flow to an error event; mxcli could, and did. It is now **MDL084**, at error severity, so `exec`'s pre-flight refuses the script with nothing written (`--no-check` still applies it, for reproducing the build failure). diff --git a/cmd/mxcli/marketplace/install_relative_dir_test.go b/cmd/mxcli/marketplace/install_relative_dir_test.go new file mode 100644 index 000000000..cb2584548 --- /dev/null +++ b/cmd/mxcli/marketplace/install_relative_dir_test.go @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: Apache-2.0 + +package marketplace + +import ( + "os" + "path/filepath" + "testing" +) + +// TestInstallPackageFiles_RelativeProjectDir: the containment guard must not +// refuse a legitimate entry just because the caller named the project by a +// relative path. `mxcli marketplace install ... -p app.mpr` hands this function +// filepath.Dir("app.mpr") == ".", and filepath.Join(".", "manifest.json") drops +// the dot — so the joined path "manifest.json" was compared against the prefix +// "./" and refused as "would write outside the project". Found on a real +// install (2026-09-20): the module had already been transplanted into the model +// when the bundled-file step failed, so the command reported failure over a +// half-finished install. +func TestInstallPackageFiles_RelativeProjectDir(t *testing.T) { + mpk := buildMPK(t, map[string]string{ + "manifest.json": `{"name":"probe"}`, + "themesource/probe/web/main.scss": "// probe", + }) + proj := t.TempDir() + wd, _ := os.Getwd() + if err := os.Chdir(proj); err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = os.Chdir(wd) }) + + written, _, err := InstallPackageFiles(mpk, ".") + if err != nil { + t.Fatalf("relative project dir refused a legitimate entry: %v", err) + } + if len(written) != 2 { + t.Fatalf("written = %v, want both entries", written) + } + for _, rel := range []string{"manifest.json", filepath.Join("themesource", "probe", "web", "main.scss")} { + if _, err := os.Stat(filepath.Join(proj, rel)); err != nil { + t.Errorf("%s not written under the project: %v", rel, err) + } + } + + // The guard itself must still hold with a relative dir. + bad := buildMPK(t, map[string]string{"../escape.txt": "x"}) + if _, _, err := InstallPackageFiles(bad, "."); err == nil { + t.Fatalf("a traversal entry must still be refused with a relative project dir") + } + if _, err := os.Stat(filepath.Join(filepath.Dir(proj), "escape.txt")); err == nil { + t.Fatalf("traversal entry escaped the project") + } +} diff --git a/cmd/mxcli/marketplace/update.go b/cmd/mxcli/marketplace/update.go index 0f09cdcb4..0aaf74814 100644 --- a/cmd/mxcli/marketplace/update.go +++ b/cmd/mxcli/marketplace/update.go @@ -317,6 +317,13 @@ func setBoolField(doc bson.D, key string, value bool) { // rather than dropped quietly, because "the package wanted a different version" // is exactly the thing that was invisible before. func InstallPackageFiles(mpkPath, projectDir string) (written []string, skipped []SkippedFile, err error) { + // Anchor the project first. A relative projectDir ("." from `-p app.mpr`) + // made filepath.Join drop the dot, so "manifest.json" was compared against + // the prefix "./" and every legitimate entry was refused as a traversal. + projectDir, err = filepath.Abs(projectDir) + if err != nil { + return nil, nil, fmt.Errorf("resolve project dir: %w", err) + } zr, err := zip.OpenReader(mpkPath) if err != nil { return nil, nil, fmt.Errorf("open package %s: %w", filepath.Base(mpkPath), err) From b549f727dcde71352d3bd10f0dc07b1c21f947c9 Mon Sep 17 00:00:00 2001 From: MendixMau Date: Sun, 20 Sep 2026 19:09:18 +0000 Subject: [PATCH 2/2] marketplace install --file: install a .mpk from disk through the same writer MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `marketplace install` could only be fed a marketplace content id, although every function the online path runs after its download — PackageProject, PerformInstall, InstallPackageFiles, moduleNameFromMpk — already took a plain path. Its own help sent anyone holding a .mpk (an internal or company module, a theme module, a package fetched with `marketplace download`, a CI job with no PAT) to Studio Pro or `mx module-import`, the two routes it explains are worse: module-import rewrites MPR v2 as v1 and refuses theme modules. --file skips the lookup and the fetch and hands the same core the path it expects. No marketplace client is constructed, so no PAT. The kind is read from the package's own package.xml: a module goes through the transplant writer, a widget is placed under widgets/, anything else is refused. --file and a content id are mutually exclusive; --version is refused with --file because it selects a marketplace release. A module from disk has no marketplace identity, so PerformInstall no longer stamps FromAppStore/AppStoreGuid when the version id is empty; `update` and `diff` then report, correctly, that no marketplace content is installed under it rather than claiming a release while naming none. Tests: eight unit tests (no mx, no network; the client factory is fatal if called) and one integration test that installs a real package into a blank project (`MXCLI_TEST_MPK`, `MXCLI_TEST_MPK_MENDIX`; skips otherwise). Field run, Linux container, no Studio Pro, mxbuild 11.14.0: a module exported with `mx create-module-package` from one 11.14.0 app installed into a fresh MPR v2 app via a relative -p — 4 units copied, 6 bundled files, mprcontents/ intact, empty Source column in SHOW MODULES, mx check 0 errors, second run reports "already installed". A Mendix 10.6.4 theme package is refused with mx's own version-window message and the app is left untouched: the reference project is still built by mx at the project's version, so mx module-import's window applies to --file exactly as to an online install. Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 8 + cmd/mxcli/cmd_marketplace.go | 6 +- cmd/mxcli/cmd_marketplace_install.go | 163 +++++++++++--- ...rketplace_install_file_integration_test.go | 163 ++++++++++++++ .../cmd_marketplace_install_file_test.go | 203 ++++++++++++++++++ cmd/mxcli/marketplace/update.go | 11 +- docs-site/src/guides/marketplace.md | 22 ++ 7 files changed, 539 insertions(+), 37 deletions(-) create mode 100644 cmd/mxcli/cmd_marketplace_install_file_integration_test.go create mode 100644 cmd/mxcli/cmd_marketplace_install_file_test.go diff --git a/CHANGELOG.md b/CHANGELOG.md index 36f6bf53a..f5557f497 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Added +- **`marketplace install --file `** — installs a `.mpk` that is already on disk through the same writer as an online install, with no marketplace lookup and no PAT. The kind is read from the package's own `package.xml`: a module goes through the transplant writer (storage format preserved, theme modules included), a widget lands in `widgets/`, anything else is refused rather than guessed at. `--file` and a content id are mutually exclusive, and `--version` is refused with `--file` because it selects a marketplace release. + + This closes the gap the command's own help used to point at: `marketplace download` could fetch a package to disk, after which the only routes were Studio Pro or `mx module-import` — the two paths the help itself explains are worse (module-import rewrites MPR v2 as v1 and refuses theme modules). Every function the online path runs after its download (`PackageProject`, `PerformInstall`, `InstallPackageFiles`, `moduleNameFromMpk`) already took a plain path; `--file` hands them one. + + A module installed from disk carries **no marketplace version stamp**, because it has no marketplace identity: `PerformInstall` skips `StampMarketplaceVersion` when the version id is empty, so `update` and `diff` report — correctly — that no marketplace content is installed under it, instead of claiming a release while naming none. + + Field run (Linux, no Studio Pro, mxbuild 11.14.0): a module exported with `mx create-module-package` from one Mendix 11.14.0 app, installed with `--file` into a fresh MPR v2 app named by a relative `-p` — 4 units copied, 6 bundled files installed, `mprcontents/` intact, `SHOW MODULES` lists the module with an empty Source column, `mx check` reports 0 errors; a second run reports "already installed" and changes nothing. A Mendix 10.6.4 theme package is refused with `mx`'s own message (the reference project is still built by `mx` at the project's version, so `mx module-import`'s window applies) and the app is left untouched. + - **Named datasources on pluggable widgets** (mendixlabs/mxcli#1109, follow-up to #643) — a datasource-typed property can be given under its own schema key (`optionsSourceAssociationDataSource: database from Module.Customer`) instead of only through the generic `datasource:` clause, so a widget exposing several datasources can be given each of them. A mode gated on `hasDataSource` is selected by a named datasource too, and the new `hasDataSource:` condition says WHICH datasource selects a mode — the legible form for a widget whose modes are exclusive by that (a ComboBox's association vs database), where bare `hasDataSource` cannot tell them apart and mode order silently decides. A dependent property binds against ITS datasource's entity: the link is the widget package's own (`widget.xml`'s `dataSource="…"`), not mapping order, so a DatagridDropdownFilter's `refCaption` resolves against `refOptions`' entity and its `attr` against `linkedDs`'. Verified end to end on a real project: both datasources persisted and the two attributes stored as `NamedDS.Order.Number` and `NamedDS.Customer.Name`. diff --git a/cmd/mxcli/cmd_marketplace.go b/cmd/mxcli/cmd_marketplace.go index 95ac3f0e1..f6f5c3bf3 100644 --- a/cmd/mxcli/cmd_marketplace.go +++ b/cmd/mxcli/cmd_marketplace.go @@ -25,8 +25,10 @@ var marketplaceCmd = &cobra.Command{ Requires a Personal Access Token (PAT). Run 'mxcli auth login' first. -'marketplace download' fetches a content version's .mpk to disk. To install a -downloaded module into a project, use Studio Pro or 'mx module-import'.`, +'marketplace download' fetches a content version's .mpk to disk; +'marketplace install --file ' installs a .mpk from disk — a downloaded +one, or any package distributed as a file — through the same writer as an online +install, with no PAT.`, } var marketplaceSearchCmd = &cobra.Command{ diff --git a/cmd/mxcli/cmd_marketplace_install.go b/cmd/mxcli/cmd_marketplace_install.go index 83a8838ee..c675e14fb 100644 --- a/cmd/mxcli/cmd_marketplace_install.go +++ b/cmd/mxcli/cmd_marketplace_install.go @@ -22,9 +22,10 @@ import ( ) var marketplaceInstallCmd = &cobra.Command{ - Use: "install ", + Use: "install | install --file ", Short: "Download and install a marketplace item into a project", - Long: `Download a marketplace content version and install it into a project. + Long: `Download a marketplace content version and install it into a project, +or install a .mpk you already have on disk with --file. Install is type-aware: - Widget copied into the project's widgets/ folder (overwrites on update) @@ -42,10 +43,19 @@ binary .mpr, one-way — and it works for theme modules, which module-import refuses outright. Everything the package ships (widgets, themesource, ...) is installed alongside the model. ---allow-format-change selects the legacy module-import path instead.`, +--allow-format-change selects the legacy module-import path instead. + +--file installs a package from disk through the same writer, with +no marketplace lookup and no PAT: the kind (module or widget) is read from the +package's own package.xml. That is the route for a module distributed as a file +— an internal or company-standard module, a theme module, a package fetched +earlier with 'marketplace download', or any CI/air-gapped environment. A module +installed this way carries no marketplace version stamp, because it has none; +'marketplace update' and 'diff' will not claim it.`, Example: ` mxcli marketplace install 20 -p app.mpr - mxcli marketplace install 2888 --version 7.0.3 -p app.mpr`, - Args: cobra.ExactArgs(1), + mxcli marketplace install 2888 --version 7.0.3 -p app.mpr + mxcli marketplace install --file ./CompanyTheme.mpk -p app.mpr`, + Args: cobra.MaximumNArgs(1), RunE: runMarketplaceInstall, // A failed install/update/diff is a runtime failure, not a misuse of the // command: printing the full flag list on top of the error buries it. @@ -61,16 +71,14 @@ func init() { marketplaceInstallCmd.Flags().String("version", "", "version number to install (default: latest)") marketplaceInstallCmd.Flags().Bool("allow-format-change", false, "use the legacy 'mx module-import' path, which rewrites an MPR v2 project as v1 (one-way)") + marketplaceInstallCmd.Flags().String("file", "", + "install a .mpk from disk instead of marketplace content (no PAT, no lookup)") _ = marketplaceInstallCmd.MarkFlagRequired("project") marketplaceCmd.AddCommand(marketplaceInstallCmd) } func runMarketplaceInstall(cmd *cobra.Command, args []string) error { - contentID, err := parseContentID(args[0]) - if err != nil { - return err - } mprPath, _ := cmd.Flags().GetString("project") if _, err := os.Stat(mprPath); err != nil { return fmt.Errorf("project not found: %s", mprPath) @@ -78,6 +86,26 @@ func runMarketplaceInstall(cmd *cobra.Command, args []string) error { versionNumber, _ := cmd.Flags().GetString("version") allowFormatChange, _ := cmd.Flags().GetBool("allow-format-change") + // A package on disk: no marketplace client is constructed at all, so no PAT + // is needed and nothing is fetched. Everything after the download step is + // shared with the online path. + if filePath, _ := cmd.Flags().GetString("file"); filePath != "" { + if len(args) != 0 { + return fmt.Errorf("give either a content id or --file, not both") + } + if versionNumber != "" { + return fmt.Errorf("--version selects a marketplace release; it does not apply to --file") + } + return installFromFile(cmd.Context(), filePath, mprPath, allowFormatChange, cmd.OutOrStdout()) + } + if len(args) != 1 { + return fmt.Errorf("a content id is required (or --file for a package on disk)") + } + contentID, err := parseContentID(args[0]) + if err != nil { + return err + } + client, err := newMarketplaceClient(cmd.Context(), cmd) if err != nil { return err @@ -128,6 +156,29 @@ func runMarketplaceInstall(cmd *cobra.Command, args []string) error { } } +// installFromFile installs a package that is already on disk. The package's +// own package.xml says what it is: a module (installed through the transplant +// writer, exactly as an online install would be) or a widget (placed under +// widgets/). Anything else is refused rather than guessed at. +func installFromFile(ctx context.Context, mpkPath, mprPath string, allowFormatChange bool, out io.Writer) error { + if _, err := os.Stat(mpkPath); err != nil { + return fmt.Errorf("package not found: %s", mpkPath) + } + pkg, err := readPackageXML(mpkPath) + if err != nil { + return fmt.Errorf("inspect package: %w", err) + } + switch { + case pkg.ModelerProject.Module.Name != "": + return installModuleFromFile(ctx, mpkPath, mprPath, allowFormatChange, "", "", + "from "+filepath.Base(mpkPath), out) + case pkg.ClientModule.Name != "": + return placeWidgetFile(mpkPath, filepath.Dir(mprPath), out) + } + return fmt.Errorf("%s is neither a module nor a widget package: its package.xml names no module and no clientModule", + filepath.Base(mpkPath)) +} + // installWidget copies the widget .mpk into the project's widgets/ folder. // An existing file with the same name is overwritten (the update path). func installWidget(ctx context.Context, client *marketplace.Client, v *marketplace.Version, projDir string, out io.Writer) error { @@ -140,10 +191,34 @@ func installWidget(ctx context.Context, client *marketplace.Client, v *marketpla return err } fmt.Fprintf(out, "Installed widget %s into %s\n", v.VersionNumber, dest) - fmt.Fprintln(out, "Run 'mxcli fix widgets -p ' (or reload in Studio Pro) to pick it up.") + printWidgetNext(out) return nil } +// placeWidgetFile is installWidget for a package already on disk: the same +// destination and the same overwrite-on-update rule, without the download. +func placeWidgetFile(mpkPath, projDir string, out io.Writer) error { + widgetsDir := filepath.Join(projDir, "widgets") + if err := os.MkdirAll(widgetsDir, 0o755); err != nil { + return fmt.Errorf("create widgets dir: %w", err) + } + body, err := os.ReadFile(mpkPath) + if err != nil { + return fmt.Errorf("read %s: %w", mpkPath, err) + } + dest := filepath.Join(widgetsDir, filepath.Base(mpkPath)) + if err := os.WriteFile(dest, body, 0o644); err != nil { + return fmt.Errorf("write %s: %w", dest, err) + } + fmt.Fprintf(out, "Installed widget %s into %s\n", filepath.Base(mpkPath), dest) + printWidgetNext(out) + return nil +} + +func printWidgetNext(out io.Writer) { + fmt.Fprintln(out, "Run 'mxcli fix widgets -p ' (or reload in Studio Pro) to pick it up.") +} + // installModule imports a module .mpk into the project, but only when the module // is not already present. An existing module is reported, not modified. func installModule(ctx context.Context, client *marketplace.Client, v *marketplace.Version, mprPath string, allowFormatChange bool, out io.Writer) error { @@ -157,6 +232,17 @@ func installModule(ctx context.Context, client *marketplace.Client, v *marketpla if err != nil { return err } + return installModuleFromFile(ctx, mpkPath, mprPath, allowFormatChange, v.VersionNumber, v.VersionID, + "version "+v.VersionNumber, out) +} + +// installModuleFromFile is the part of a module install that starts once the +// package is on disk — shared by the online path (after its download) and by +// --file (which has no download). versionNumber and versionID are the +// marketplace identity to record on the module; both are empty for a package +// from disk, which has none. label is how the package is named in output. +func installModuleFromFile(ctx context.Context, mpkPath, mprPath string, allowFormatChange bool, + versionNumber, versionID, label string, out io.Writer) error { moduleName, err := moduleNameFromMpk(mpkPath) if err != nil { @@ -175,7 +261,7 @@ func installModule(ctx context.Context, client *marketplace.Client, v *marketpla if existing { // Postponed: do NOT auto-update modules — see the module-update memory. fmt.Fprintf(out, "Module %q is already installed (version %s).\n", moduleName, displayVer(installedVer)) - fmt.Fprintf(out, "Target version: %s.\n", v.VersionNumber) + fmt.Fprintf(out, "Target: %s.\n", label) fmt.Fprintln(out, "In-place module updates are not applied automatically (they can discard local") fmt.Fprintln(out, "edits and change persistent-entity IDs, which loses data). Update via Studio Pro.") return nil @@ -185,12 +271,12 @@ func installModule(ctx context.Context, client *marketplace.Client, v *marketpla // the project's storage format and works for theme modules. --allow-format-change // selects the legacy `mx module-import`, which does neither. if !allowFormatChange { - res, ierr := installByTransplant(ctx, mpkPath, mprPath, moduleName, mendixVer, v) + res, ierr := installByTransplant(ctx, mpkPath, mprPath, moduleName, mendixVer, versionNumber, versionID) if ierr != nil { return ierr } - fmt.Fprintf(out, "Installed module %q version %s into %s\n", - moduleName, v.VersionNumber, filepath.Base(mprPath)) + fmt.Fprintf(out, "Installed module %q %s into %s\n", + moduleName, label, filepath.Base(mprPath)) fmt.Fprintf(out, " %d units copied, %d bundled file(s) installed.\n", res.UnitsCopied, len(res.FilesInstalled)) reportSkippedFiles(out, res.FilesSkipped) @@ -216,7 +302,7 @@ func installModule(ctx context.Context, client *marketplace.Client, v *marketpla if runErr != nil { return fmt.Errorf("mx module-import failed: %w\n%s", runErr, strings.TrimSpace(string(combined))) } - fmt.Fprintf(out, "Imported module %q version %s into %s\n", moduleName, v.VersionNumber, filepath.Base(mprPath)) + fmt.Fprintf(out, "Imported module %q %s into %s\n", moduleName, label, filepath.Base(mprPath)) reportFormatChange(mprPath, out) return nil } @@ -224,7 +310,7 @@ func installModule(ctx context.Context, client *marketplace.Client, v *marketpla // installByTransplant builds a reference project from the package and copies the // module out of it, so the destination keeps its MPR format. func installByTransplant(ctx context.Context, mpkPath, mprPath, moduleName, mendixVer string, - v *marketplace.Version) (*mp.UpdateResult, error) { + versionNumber, versionID string) (*mp.UpdateResult, error) { work, err := os.MkdirTemp("", "mxinstall") if err != nil { @@ -240,7 +326,7 @@ func installByTransplant(ctx context.Context, mpkPath, mprPath, moduleName, mend if err != nil { return nil, fmt.Errorf("build a reference project from the package: %w", err) } - return mp.PerformInstall(mprPath, refMpr, mpkPath, moduleName, v.VersionNumber, v.VersionID, newBackendFactory()) + return mp.PerformInstall(mprPath, refMpr, mpkPath, moduleName, versionNumber, versionID, newBackendFactory()) } // isMPRv2 reports whether the project at mprPath uses the MPR v2 storage format: @@ -360,12 +446,14 @@ type mpkPackageXML struct { } `xml:"modelerProject"` } -// moduleNameFromMpk reads package.xml from the .mpk and returns the module name -// (from for modules, or for widgets). -func moduleNameFromMpk(mpkPath string) (string, error) { +// readPackageXML returns the parsed package.xml of a .mpk. It is the one place +// that knows where a package describes itself, so both "what is its name" and +// "what kind of package is it" read from here. +func readPackageXML(mpkPath string) (mpkPackageXML, error) { + var pkg mpkPackageXML zr, err := zip.OpenReader(mpkPath) if err != nil { - return "", err + return pkg, err } defer zr.Close() for _, f := range zr.File { @@ -374,24 +462,33 @@ func moduleNameFromMpk(mpkPath string) (string, error) { } rc, err := f.Open() if err != nil { - return "", err + return pkg, err } data, err := io.ReadAll(rc) _ = rc.Close() if err != nil { - return "", err + return pkg, err } - var pkg mpkPackageXML if err := xml.Unmarshal(data, &pkg); err != nil { - return "", err - } - if pkg.ModelerProject.Module.Name != "" { - return pkg.ModelerProject.Module.Name, nil + return pkg, err } - if pkg.ClientModule.Name != "" { - return pkg.ClientModule.Name, nil - } - return "", fmt.Errorf("package.xml has no module name") + return pkg, nil + } + return pkg, fmt.Errorf("no package.xml in %s", filepath.Base(mpkPath)) +} + +// moduleNameFromMpk reads package.xml from the .mpk and returns the module name +// (from for modules, or for widgets). +func moduleNameFromMpk(mpkPath string) (string, error) { + pkg, err := readPackageXML(mpkPath) + if err != nil { + return "", err + } + if pkg.ModelerProject.Module.Name != "" { + return pkg.ModelerProject.Module.Name, nil + } + if pkg.ClientModule.Name != "" { + return pkg.ClientModule.Name, nil } - return "", fmt.Errorf("no package.xml in %s", filepath.Base(mpkPath)) + return "", fmt.Errorf("package.xml has no module name") } diff --git a/cmd/mxcli/cmd_marketplace_install_file_integration_test.go b/cmd/mxcli/cmd_marketplace_install_file_integration_test.go new file mode 100644 index 000000000..7cbe52524 --- /dev/null +++ b/cmd/mxcli/cmd_marketplace_install_file_integration_test.go @@ -0,0 +1,163 @@ +//go:build integration + +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "archive/zip" + "bytes" + "context" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + + modelsdk "github.com/mendixlabs/mxcli" + "github.com/mendixlabs/mxcli/cmd/mxcli/docker" + "github.com/mendixlabs/mxcli/internal/marketplace" + "github.com/spf13/cobra" +) + +// TestInstallFile_ModuleEndToEnd installs a real module package from disk into a +// blank project and checks what an online install promises: the module is in +// the model, the project's storage format is untouched, the package's bundled +// files are on disk, and — new for --file — the module carries no marketplace +// stamp, because it has no marketplace identity. +// +// It needs a package and an mxbuild: +// +// MXCLI_TEST_MPK=/path/to/Module.mpk MXCLI_TEST_MPK_MENDIX=11.14.0 \ +// go test -tags integration ./cmd/mxcli -run InstallFile_ModuleEndToEnd -count=1 -v +// +// Without them it skips, so the suite stays hermetic by default. The package +// must NOT be one the blank template already ships (Administration, Atlas_*, +// DataWidgets, MyFirstModule, …): that exercises the "already installed" branch +// instead, which the second half of this test covers on purpose. +func TestInstallFile_ModuleEndToEnd(t *testing.T) { + mpk := os.Getenv("MXCLI_TEST_MPK") + version := os.Getenv("MXCLI_TEST_MPK_MENDIX") + if mpk == "" || version == "" { + t.Skip("set MXCLI_TEST_MPK and MXCLI_TEST_MPK_MENDIX to run this") + } + if _, err := os.Stat(mpk); err != nil { + t.Skipf("MXCLI_TEST_MPK does not exist: %v", err) + } + mxPath, err := docker.ResolveMxForVersion("", version) + if err != nil { + t.Skipf("mxbuild %s is not cached; run 'mxcli setup mxbuild --version %s'", version, version) + } + moduleName, err := moduleNameFromMpk(mpk) + if err != nil { + t.Fatalf("read the package's module name: %v", err) + } + + // A blank project at the requested version. Short temp path on purpose: mx + // create-project fails under a deep directory (see scratch.go). + work, err := os.MkdirTemp("", "mxif") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { os.RemoveAll(work) }) + create := exec.CommandContext(context.Background(), mxPath, "create-project", "--app-name", "InstallFileTarget") + create.Dir = work + docker.PrepareMxCommand(create) + if out, err := create.CombinedOutput(); err != nil { + t.Fatalf("mx create-project: %v\n%s", err, out) + } + var mprPath string + _ = filepath.WalkDir(work, func(p string, d os.DirEntry, _ error) error { + if !d.IsDir() && strings.HasSuffix(p, ".mpr") && mprPath == "" { + mprPath = p + } + return nil + }) + if mprPath == "" { + t.Fatal("no .mpr in the blank project") + } + // ResolveMxForVersion falls back to any cached mxbuild; refuse to test + // against a project that is not at the version we asked for. + if got := mendixVersionOf(mprPath); got != version { + t.Skipf("blank project is Mendix %s, wanted %s (cached mxbuild fallback)", got, version) + } + wasV2 := isMPRv2(mprPath) + + // No marketplace client may be constructed. HOME is deliberately NOT + // redirected here (unlike the unit tests): the transplant resolves mx from + // ~/.mxcli/mxbuild, and hiding that behind a temp HOME makes the install + // fail with "mx not found" for a reason that has nothing to do with --file. + origFactory := marketplaceClientFactory + marketplaceClientFactory = func(_ context.Context, _ *cobra.Command) (*marketplace.Client, error) { + t.Fatal("--file must not construct a marketplace client") + return nil, nil + } + t.Cleanup(func() { marketplaceClientFactory = origFactory }) + + run := func() (string, error) { + resetMarketplaceFlags() + var out bytes.Buffer + rootCmd.SetOut(&out) + rootCmd.SetErr(&out) + rootCmd.SetArgs([]string{"marketplace", "install", "--file", mpk, "-p", mprPath}) + // Two statements on purpose: return operands are evaluated left to + // right, so `return out.String(), rootCmd.Execute…()` reads the buffer + // before the command has written to it. + err := rootCmd.ExecuteContext(context.Background()) + return out.String(), err + } + + out, err := run() + if err != nil { + t.Fatalf("install --file: %v\n%s", err, out) + } + if !strings.Contains(out, "Installed module") { + t.Fatalf("expected an install report, got:\n%s", out) + } + + // 1. The module is in the model, and carries no marketplace stamp. + reader, err := modelsdk.Open(mprPath) + if err != nil { + t.Fatalf("open project after install: %v", err) + } + found, appStoreVersion := findModule(reader, moduleName) + _ = reader.Disconnect() + if !found { + t.Fatalf("module %q is not in the project after install\n%s", moduleName, out) + } + if appStoreVersion != "" { + t.Errorf("a package from disk must not be stamped with a marketplace version, got %q", appStoreVersion) + } + + // 2. Storage format preserved — the reason the transplant writer exists. + if wasV2 && !isMPRv2(mprPath) { + t.Errorf("project was MPR v2 before the install and is not afterwards") + } + + // 3. Bundled files landed. Pick any non-manifest entry from the package. + zr, err := zip.OpenReader(mpk) + if err != nil { + t.Fatal(err) + } + defer zr.Close() + checked := 0 + for _, f := range zr.File { + if f.FileInfo().IsDir() || f.Name == "package.xml" || f.Name == "project.mpr" || f.Name == "manifest.json" { + continue + } + if _, serr := os.Stat(filepath.Join(filepath.Dir(mprPath), filepath.FromSlash(f.Name))); serr != nil { + t.Errorf("bundled file %s not installed: %v", f.Name, serr) + } + checked++ + } + t.Logf("module %q installed from %s: %d bundled file(s) verified on disk", moduleName, filepath.Base(mpk), checked) + + // 4. A second install is reported, not repeated. + out, err = run() + if err != nil { + t.Fatalf("second install --file: %v\n%s", err, out) + } + if !strings.Contains(out, "already installed") { + t.Errorf("second install should report the module as already installed, got:\n%s", out) + } +} diff --git a/cmd/mxcli/cmd_marketplace_install_file_test.go b/cmd/mxcli/cmd_marketplace_install_file_test.go new file mode 100644 index 000000000..ff2a4382a --- /dev/null +++ b/cmd/mxcli/cmd_marketplace_install_file_test.go @@ -0,0 +1,203 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "archive/zip" + "bytes" + "context" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/mendixlabs/mxcli/internal/marketplace" + "github.com/spf13/cobra" +) + +// buildLocalMPK writes a .mpk (a zip) with the given entries and returns its path. +func buildLocalMPK(t *testing.T, name string, entries map[string]string) string { + t.Helper() + p := filepath.Join(t.TempDir(), name) + f, err := os.Create(p) + if err != nil { + t.Fatal(err) + } + zw := zip.NewWriter(f) + for entry, body := range entries { + w, err := zw.Create(entry) + if err != nil { + t.Fatal(err) + } + if _, err := w.Write([]byte(body)); err != nil { + t.Fatal(err) + } + } + if err := zw.Close(); err != nil { + t.Fatal(err) + } + if err := f.Close(); err != nil { + t.Fatal(err) + } + return p +} + +// runInstallFile runs `marketplace install` with a client factory that FAILS the +// test if it is ever called: a --file install must never construct a marketplace +// client, because that is what needs a PAT. The project directory holds a +// placeholder .mpr — the widget path only checks that the project exists. +func runInstallFile(t *testing.T, args ...string) (string, error) { + t.Helper() + t.Setenv("HOME", t.TempDir()) + + origFactory := marketplaceClientFactory + marketplaceClientFactory = func(_ context.Context, _ *cobra.Command) (*marketplace.Client, error) { + t.Fatal("--file must not construct a marketplace client (that is what needs a PAT)") + return nil, nil + } + t.Cleanup(func() { marketplaceClientFactory = origFactory }) + + resetMarketplaceFlags() + + var out bytes.Buffer + rootCmd.SetOut(&out) + rootCmd.SetErr(&out) + rootCmd.SetArgs(append([]string{"marketplace", "install"}, args...)) + err := rootCmd.ExecuteContext(context.Background()) + return out.String(), err +} + +func placeholderProject(t *testing.T) string { + t.Helper() + dir := t.TempDir() + mpr := filepath.Join(dir, "app.mpr") + if err := os.WriteFile(mpr, []byte("placeholder"), 0o644); err != nil { + t.Fatal(err) + } + return mpr +} + +const widgetPackageXML = ` + + + + +` + +func TestInstallFile_WidgetLandsInWidgetsDir(t *testing.T) { + mpr := placeholderProject(t) + src := buildLocalMPK(t, "Badge.mpk", map[string]string{ + "package.xml": widgetPackageXML, + "Badge.xml": "", + }) + + out, err := runInstallFile(t, "--file", src, "-p", mpr) + if err != nil { + t.Fatalf("run: %v\noutput: %s", err, out) + } + dest := filepath.Join(filepath.Dir(mpr), "widgets", "Badge.mpk") + got, rerr := os.ReadFile(dest) + if rerr != nil { + t.Fatalf("widget not placed at %s: %v\noutput: %s", dest, rerr, out) + } + want, _ := os.ReadFile(src) + if !bytes.Equal(got, want) { + t.Errorf("widget bytes differ from the source package") + } + if !strings.Contains(out, "Installed widget Badge.mpk") { + t.Errorf("expected an install line naming the file, got: %s", out) + } + if !strings.Contains(out, "mxcli fix widgets") { + t.Errorf("expected the fix-widgets hint, got: %s", out) + } +} + +func TestInstallFile_WidgetOverwritesOnReinstall(t *testing.T) { + mpr := placeholderProject(t) + dest := filepath.Join(filepath.Dir(mpr), "widgets", "Badge.mpk") + if err := os.MkdirAll(filepath.Dir(dest), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(dest, []byte("stale"), 0o644); err != nil { + t.Fatal(err) + } + src := buildLocalMPK(t, "Badge.mpk", map[string]string{"package.xml": widgetPackageXML}) + if out, err := runInstallFile(t, "--file", src, "-p", mpr); err != nil { + t.Fatalf("run: %v\noutput: %s", err, out) + } + got, _ := os.ReadFile(dest) + if string(got) == "stale" { + t.Errorf("reinstall must overwrite the existing widget (the update path)") + } +} + +func TestInstallFile_RefusesContentIDAndFileTogether(t *testing.T) { + mpr := placeholderProject(t) + src := buildLocalMPK(t, "x.mpk", map[string]string{"package.xml": widgetPackageXML}) + _, err := runInstallFile(t, "20", "--file", src, "-p", mpr) + if err == nil || !strings.Contains(err.Error(), "either a content id or --file") { + t.Fatalf("expected the either/or refusal, got: %v", err) + } +} + +func TestInstallFile_RefusesVersionFlag(t *testing.T) { + mpr := placeholderProject(t) + src := buildLocalMPK(t, "x.mpk", map[string]string{"package.xml": widgetPackageXML}) + _, err := runInstallFile(t, "--file", src, "--version", "1.0.0", "-p", mpr) + if err == nil || !strings.Contains(err.Error(), "--version") { + t.Fatalf("expected --version to be refused with --file, got: %v", err) + } +} + +func TestInstallFile_MissingPackage(t *testing.T) { + mpr := placeholderProject(t) + _, err := runInstallFile(t, "--file", filepath.Join(t.TempDir(), "nope.mpk"), "-p", mpr) + if err == nil || !strings.Contains(err.Error(), "package not found") { + t.Fatalf("expected 'package not found', got: %v", err) + } +} + +func TestInstallFile_RefusesUnknownPackageKind(t *testing.T) { + mpr := placeholderProject(t) + src := buildLocalMPK(t, "odd.mpk", map[string]string{ + "package.xml": ``, + }) + _, err := runInstallFile(t, "--file", src, "-p", mpr) + if err == nil || !strings.Contains(err.Error(), "neither a module nor a widget") { + t.Fatalf("expected the unknown-kind refusal, got: %v", err) + } + if _, serr := os.Stat(filepath.Join(filepath.Dir(mpr), "widgets")); serr == nil { + t.Errorf("a refused package must not create widgets/") + } +} + +func TestInstallFile_NoContentIDAndNoFile(t *testing.T) { + mpr := placeholderProject(t) + _, err := runInstallFile(t, "-p", mpr) + if err == nil || !strings.Contains(err.Error(), "--file") { + t.Fatalf("expected the usage error to mention --file, got: %v", err) + } +} + +// The online path must be untouched by the refactor: a content id still parses +// and still reaches the client factory (which this helper makes fatal). +func TestInstallFile_ContentIDStillUsesTheClient(t *testing.T) { + mpr := placeholderProject(t) + t.Setenv("HOME", t.TempDir()) + called := false + origFactory := marketplaceClientFactory + marketplaceClientFactory = func(_ context.Context, _ *cobra.Command) (*marketplace.Client, error) { + called = true + return nil, context.Canceled + } + t.Cleanup(func() { marketplaceClientFactory = origFactory }) + resetMarketplaceFlags() + var out bytes.Buffer + rootCmd.SetOut(&out) + rootCmd.SetErr(&out) + rootCmd.SetArgs([]string{"marketplace", "install", "20", "-p", mpr}) + _ = rootCmd.ExecuteContext(context.Background()) + if !called { + t.Fatalf("a content-id install must still go through the marketplace client") + } +} diff --git a/cmd/mxcli/marketplace/update.go b/cmd/mxcli/marketplace/update.go index 0aaf74814..41f7131fe 100644 --- a/cmd/mxcli/marketplace/update.go +++ b/cmd/mxcli/marketplace/update.go @@ -432,8 +432,15 @@ func PerformInstall(mprPath, referenceMpr, packageMpk, moduleName, version, vers if err != nil { return nil, fmt.Errorf("copy the module in: %w", err) } - if err := StampMarketplaceVersion(mprPath, moduleName, version, versionID); err != nil { - return nil, fmt.Errorf("record the installed version: %w", err) + // A package installed from disk (marketplace install --file) has no + // marketplace identity. Stamping it FromAppStore with an empty GUID would + // tell `update` and `diff` that some release is installed while naming + // none; leaving the module unstamped is the truthful record, and both then + // report — correctly — that no marketplace content is installed under it. + if versionID != "" { + if err := StampMarketplaceVersion(mprPath, moduleName, version, versionID); err != nil { + return nil, fmt.Errorf("record the installed version: %w", err) + } } files, skippedFiles, err := InstallPackageFiles(packageMpk, filepath.Dir(mprPath)) if err != nil { diff --git a/docs-site/src/guides/marketplace.md b/docs-site/src/guides/marketplace.md index 69d40b353..ef1f957d2 100644 --- a/docs-site/src/guides/marketplace.md +++ b/docs-site/src/guides/marketplace.md @@ -65,6 +65,28 @@ mxcli marketplace install 2888 --version 7.0.3 -p app.mpr # a module | **Module** (already present) | **Reported, not modified** — see below. | | Theme / Starter App / Sample | Downloaded to disk with import instructions (import via Studio Pro). | +### Installing a `.mpk` you already have (`--file`) + +Not every package comes from the marketplace: an internal or company-standard module handed +over as a file, a theme module, a package fetched earlier with `download`, or a CI job with no +token. `--file` installs such a package through the same writer as an online install, with no +marketplace lookup and no PAT: + +```bash +mxcli marketplace install --file ./CompanyTheme.mpk -p app.mpr +``` + +The kind is read from the package's own `package.xml` — a module goes through the transplant +writer (storage format preserved, theme modules included), a widget lands in `widgets/`. +Anything else is refused rather than guessed at. A module installed this way carries **no +marketplace version stamp**, because it has none: `update` and `diff` will report that no +marketplace content is installed under it, which is the truth. + +The reference project the writer builds is still created with `mx` at the project's Mendix +version, so a package that `mx` itself cannot open — one exported from a Studio Pro further back +than the version window `mx module-import` accepts — is refused with `mx`'s own message. Convert +it once with `mx convert` on a toolset whose window covers it, then `--file` the result. + ### The latest version is often not installable New releases are published against the newest Studio Pro patch within days of it shipping, and `install` with no `--version` resolves to the latest — so on a project that is not on the very newest patch, the default is routinely the one version that cannot be imported. Measured on an 11.12.1 project: the latest release of all six agent-editor stack modules required 11.12.2, published five days earlier.