Repository navigation
feat(models): remove postprocess_models.py and emit pure OOTB Pydantic DTOs #103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -92,16 +92,45 @@ both `ucp_sdk.models` and `ucp_sdk.models.schemas`: | |
|
|
||
| ### Validation | ||
|
|
||
| All models support Pydantic validation and serialization: | ||
| Pydantic models in `ucp_sdk.models` provide Tier 1 wire DTO structural typing | ||
| and tolerant-reader deserialization (preserving unknown extension fields for | ||
| forward compatibility) rather than full UCP protocol conformance. | ||
|
|
||
| To validate inbound messages against normative UCP constraints (capability | ||
| profile composition, directional `ucp_request` / `ucp_response` visibility, | ||
| `dependentRequired`, `if`/`then`, `uniqueItems`, `contains`, `propertyNames`, | ||
| `not.enum`, and closed-schema rules), verify received JSON payloads with | ||
| [`ucp-schema validate`](https://crates.io/crates/ucp-schema) prior to | ||
| deserializing into SDK DTOs: | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit (non-blocking): Worth adding a brief note here that open discriminated unions (e.g., |
||
|
|
||
| ```bash | ||
| # Self-describing response or JSON-RPC envelope | ||
| ucp-schema validate payload.json --op read | ||
|
|
||
| # REST request against a discovery profile (or --schema checkout.json --request --op create) | ||
| ucp-schema validate payload.json --profile profile.json --op create | ||
| ``` | ||
|
|
||
| ```python | ||
| from pathlib import Path | ||
| import subprocess | ||
|
|
||
| from pydantic import ValidationError | ||
| from ucp_sdk.models import Checkout | ||
|
|
||
| # Validate data against UCP schemas | ||
| payload_path = Path("payload.json") | ||
|
|
||
| # Step 1: Verify normative UCP protocol constraints with ucp-schema validate | ||
| subprocess.run( | ||
| ["ucp-schema", "validate", str(payload_path), "--op", "read"], | ||
| check=True, | ||
| ) | ||
|
|
||
| # Step 2: Deserialize verified JSON into typed Pydantic DTOs | ||
| try: | ||
| checkout = Checkout.model_validate(checkout_data) | ||
| # Serialize to JSON-compatible dict | ||
| checkout = Checkout.model_validate_json( | ||
| payload_path.read_text(encoding="utf-8") | ||
| ) | ||
| checkout_dict = checkout.model_dump(exclude_none=True) | ||
| except ValidationError as e: | ||
| print(e.errors()) | ||
|
|
@@ -145,7 +174,7 @@ a branch or commit SHA, or a path to a local UCP repository checkout. | |
| If no argument is specified, the `main` branch of the | ||
| [UCP repo](https://github.com/Universal-Commerce-Protocol/ucp) will be used. | ||
|
|
||
| The generated code is automatically post-processed and formatted using `ruff`. | ||
| The generated code is automatically formatted using `ruff`. | ||
|
|
||
| ### Running Tests | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -137,8 +137,7 @@ echo "Compiling type bundle with ucp-schema generate-types..." | |
| "$UCP_SCHEMA_BIN" generate-types --schema-dir "$SCOPED_SCHEMA_DIR" --output "$TMP_TYPES_JSON" | ||
|
|
||
| echo "Generating Pydantic models from compiled type bundle..." | ||
| rm -rf "$OUTPUT_DIR" | ||
| mkdir -p "$OUTPUT_DIR" | ||
| mkdir -p "$OUTPUT_DIR" && rm -f "$OUTPUT_DIR/_generated.py" | ||
|
|
||
| uv run \ | ||
| --link-mode=copy \ | ||
|
|
@@ -161,8 +160,51 @@ uv run \ | |
| --use-title-as-name \ | ||
| --additional-imports pydantic.ConfigDict | ||
|
|
||
| echo "Post-processing generated models (constraints the generator ignores)..." | ||
| uv run python postprocess_models.py "$TMP_TYPES_JSON" src/ucp_sdk/models "$SCOPED_SCHEMA_DIR" | ||
| echo "Generating lazy package index in $OUTPUT_DIR/__init__.py..." | ||
| python3 - <<'PY' | ||
| import ast | ||
| from pathlib import Path | ||
|
|
||
| _SYNTHETIC_WRAPPERS = {"UCPSchemaTypes", "UcpSchemaTypes"} | ||
| schemas_dir = Path("src/ucp_sdk/models/schemas") | ||
| tree = ast.parse((schemas_dir / "_generated.py").read_text(encoding="utf-8")) | ||
| type_alias_cls = getattr(ast, "TypeAlias", ()) | ||
| names: set[str] = set() | ||
| for node in tree.body: | ||
| if isinstance(node, ast.ClassDef): | ||
| names.add(node.name) | ||
| elif isinstance(node, ast.Assign): | ||
| names.update(t.id for t in node.targets if isinstance(t, ast.Name)) | ||
| elif isinstance(node, ast.AnnAssign) and isinstance(node.target, ast.Name): | ||
| names.add(node.target.id) | ||
| elif isinstance(node, type_alias_cls) and isinstance(getattr(node, "name", None), ast.Name): | ||
| names.add(node.name.id) | ||
| exports = sorted(n for n in names if not n.startswith("_") and n not in _SYNTHETIC_WRAPPERS) | ||
| imports = ", ".join(exports) | ||
| all_items = ", ".join(f'"{n}"' for n in exports) | ||
| (schemas_dir / "__init__.py").write_text( | ||
| '# generated by generate_models.sh\n\n' | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Nit: Including the Apache 2.0 license header at the top of generated |
||
| '"""Generated UCP schema models."""\n\n' | ||
| 'from __future__ import annotations\n\n' | ||
| 'import importlib\n' | ||
| 'from typing import TYPE_CHECKING, Any\n\n' | ||
| f'if TYPE_CHECKING:\n from ._generated import ({imports},)\n\n' | ||
| f'__all__ = [{all_items},]\n\n\n' | ||
| 'def __getattr__(name: str) -> Any:\n' | ||
| ' """Import public schema models and type aliases on first access."""\n' | ||
| ' if name not in __all__:\n' | ||
| ' msg = f"module {__name__!r} has no attribute {name!r}"\n' | ||
| ' raise AttributeError(msg)\n' | ||
| ' module = importlib.import_module("._generated", __name__)\n' | ||
| ' value = getattr(module, name)\n' | ||
| ' globals()[name] = value\n' | ||
| ' return value\n\n\n' | ||
| 'def __dir__() -> list[str]:\n' | ||
| ' """Return module attributes including lazily re-exported schema names."""\n' | ||
| ' return sorted({*globals(), *__all__})\n', | ||
| encoding="utf-8", | ||
| ) | ||
| PY | ||
|
|
||
| echo "Formatting generated models..." | ||
| uv run ruff check --fix src/ucp_sdk/models | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two notes here:
model-driftcurrently fails with HTTP 404 becauseUniversal-Commerce-Protocol/ucp-schemav2.0.0hasn't been published yet (waiting onUniversal-Commerce-Protocol/ucp-schema#96andUniversal-Commerce-Protocol/ucp-schema#97to merge andv2.0.0to be tagged)..github/workflows/*.ymldeclarative with single-linerun:steps, consider moving theucp-schemarelease binary download +SHA256SUMSverification into a dedicated script (orgenerate_models.sh), and removing the duplicateNormalize file endingsinline Python step below (lines 78–88), sincegenerate_models.shalready normalizes file endings onsrc/ucp_sdk/models.