Repository navigation
Conversation
A process is identified by the SHA-256 of a canonical form defined over the parsed JSON document, not over any CWL library's output, so that it is deterministic, stable across library upgrades and reproducible by any tool. Ids, doc and label where they are metadata, $namespaces and dirac: hints are dropped; prefixes are expanded to full URIs; lists become maps; fully scoped identifiers are shortened against their scope; type shorthands are expanded and named types written by short name. Each child run body is replaced by its own identity, so a workflow's identity covers its children's (a Merkle tree). pack and unpack convert to and from a self-contained $graph whose components are named by their identity. Documents must be resolved before canonicalisation: $import, $include and any other schema-salad directive is refused, since the identity would not cover what it names and a runner would fetch it. Standard library only. Closes DIRACGrid#1106 Part of DIRACGrid#1080
Unit tests ran on SQLite only, but production runs on MySQL and MariaDB, which differ from SQLite where the transformation tables rely on them. The diracx-db transformation tests now share one fixture that runs each test on SQLite and on every server given by DIRACX_TEST_MYSQL_URL or DIRACX_TEST_MARIADB_URL, and a CI job runs them on MySQL 8.4 and MariaDB 11.4 service containers, pinned by digest. It found one difference already: a failed CHECK constraint is an IntegrityError on SQLite but an OperationalError on MySQL and MariaDB through PyMySQL, so the deltas test now accepts either, matched on the constraint's name. Closes DIRACGrid#1108
…onDB insert_processes stores documents under their content hash with INSERT IGNORE, so concurrent submissions of a shared process are safe, and get_processes fetches them, on the DX-ADR-004 Processes table unchanged. MySQL's JSON type re-sorts keys and reformats numbers, so a stored row is not the bytes that were hashed: what must hold is that the value read back re-hashes to its identity, which the round-trip test checks on SQLite, MySQL and MariaDB. Closes DIRACGrid#1109 Part of DIRACGrid#1080
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1109. Part of #1080.
Blocked by #1114 and #1115, whose commits this branch includes until they merge.