Repository navigation
Conversation
A process is identified by the SHA-256 of a canonical form defined over the parsed JSON document, not over any CWL library's output, so that it is deterministic, stable across library upgrades and reproducible by any tool. Ids, doc and label where they are metadata, $namespaces and dirac: hints are dropped; prefixes are expanded to full URIs; lists become maps; fully scoped identifiers are shortened against their scope; type shorthands are expanded and named types written by short name. Each child run body is replaced by its own identity, so a workflow's identity covers its children's (a Merkle tree). pack and unpack convert to and from a self-contained $graph whose components are named by their identity. Documents must be resolved before canonicalisation: $import, $include and any other schema-salad directive is refused, since the identity would not cover what it names and a runner would fetch it. Standard library only. Closes DIRACGrid#1106 Part of DIRACGrid#1080
CWL is written in YAML, but services only ever receive JSON: the CLI resolves a document and submits the result. load_document reads YAML 1.2, as runners do, so a value such as yes stays the string a user tested, and inlines every step's run path, $import, $include and #fragment into one value. It refuses reference cycles, deep nesting, oversized files and anything but local files, and keeps references to stored processes (proc:sha256-<hex>). ruamel.yaml is a diracx-cli dependency only. The MCSimulation workgraph of DIRACGrid#1080 is a fixture: resolving the same files from another directory gives the same identities, and an edit to the shared tool changes every transformation using it. Closes DIRACGrid#1112
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1112.
Blocked by #1114, whose commit this branch includes until it merges. To merge with the
diracsubmission command that uses it.