PD-202 improve metadata functions by mfshao · Pull Request #317 · uc-cdis/gen3sdk-python

mfshao · 2026-09-30T21:34:59Z

Link to JIRA ticket if there is one: https://ctds-planx.atlassian.net/browse/PD-202

New Features

Breaking Changes

Bug Fixes

Improvements

gen3.metadata.Gen3Metadata.query() no longer requires a query parameter, if omitted, all metadata record with MDS will be returned (subject to limit and offset)
gen3.metadata.Gen3Metadata() no longer requires a gen3.auth.Gen3Auth object during initialization. If functions that requires auth will be called, an message will be posted to the user and the call will return directly

Dependency updates

Deployment changes

github-actions · 2026-09-30T21:35:39Z

The style in this PR agrees with black. ✔️

This formatting comment was generated automatically by a script in uc-cdis/wool.

github-actions · 2026-09-30T22:01:31Z

Integration Tests

Test summary after running integration tests

filepath	passed	failed	skipped	SUBTOTAL
tests/test_graph_submit_and_query.py	12	1	1	14
tests/test_data_upload.py	8	0	1	9
tests/test_presigned_url.py	8	0	0	8
tests/test_centralized_auth.py	5	0	0	5
tests/test_audit_service.py	1	0	0	1
tests/test_dbgap.py	4	0	1	5
tests/test_gen3_sdk.py	1	0	0	1
tests/test_ras_passport.py	0	0	2	2
tests/test_google_data_access.py	0	1	0	1
TOTAL	39	2	5	46

Test summary after rerunning failed integration tests

filepath	passed	failed	SUBTOTAL
tests/test_graph_submit_and_query.py	1	0	1
tests/test_google_data_access.py	0	1	1
TOTAL	1	1	2

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

github-actions · 2026-09-30T22:01:45Z

Failure Analysis

Please find the detailed test analysis report here

github-actions · 2026-10-01T22:12:28Z

Integration Tests

Test summary after running integration tests

filepath	passed	failed	skipped	SUBTOTAL
tests/test_graph_submit_and_query.py	12	1	1	14
tests/test_data_upload.py	8	0	1	9
tests/test_presigned_url.py	8	0	0	8
tests/test_centralized_auth.py	5	0	0	5
tests/test_dbgap.py	4	0	1	5
tests/test_audit_service.py	1	0	0	1
tests/test_google_data_access.py	1	0	0	1
tests/test_gen3_sdk.py	1	0	0	1
tests/test_ras_passport.py	0	0	2	2
TOTAL	40	1	5	46

Test summary after rerunning failed integration tests

filepath	passed	SUBTOTAL
tests/test_graph_submit_and_query.py	1	1
TOTAL	1	1

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

SpencerAxelrod · 2026-10-05T18:33:41Z

        return_full_metadata=False,
        limit=10,
        offset=0,
-        use_agg_mds=False,


what will happen for calls like this that still have use_agg_mds?

gen3sdk-python/gen3/tools/metadata/discovery.py

Lines 164 to 169 in fd1ad6f

partial_metadata = mds.query(

f"_guid_type={guid_type}",

return_full_metadata=True,

limit=min(limit, max_guids_per_request),

offset=offset,

use_agg_mds=use_agg_mds,

use_agg_mds will go into **kwargs and then append_query_params() will add to the request url, and I think breaks

good catch, i removed this parameter since it is not used

SpencerAxelrod · 2026-10-05T18:55:37Z

@@ -588,12 +654,14 @@ def delete_alias(self, guid, alias, **kwargs):
        return response.json()


MDS DELETE /metadata/{guid}/aliases/{alias} returns 204 (with no json response body allowed), so response.json() errors. The removed version of delete_alias() returned response.text. I think _mock_delete_request in test_metadata.py needs to be fixed to catch this

SpencerAxelrod · 2026-10-05T19:07:34Z

per MDS source:
https://github.com/uc-cdis/metadata-service/blob/eefaef4cd18ca3908684d7d5fd8ec6adae3db7f1/src/mds/aliases.py#L115-L121

So this should be 204..
which actually contradicts the API spec which says it returns 200 on successful delete alias!
https://github.com/uc-cdis/metadata-service/blob/eefaef4cd18ca3908684d7d5fd8ec6adae3db7f1/docs/openapi.yaml#L768

good catch, i updated gen3sdk code and opened this PR to update MDS openapi doc

github-actions · 2026-10-05T21:50:40Z

Integration Tests

Test summary after running integration tests

filepath	passed	failed	skipped	SUBTOTAL
tests/test_graph_submit_and_query.py	12	1	1	14
tests/test_data_upload.py	8	0	1	9
tests/test_dbgap.py	4	0	1	5
tests/test_centralized_auth.py	5	0	0	5
tests/test_presigned_url.py	8	0	0	8
tests/test_audit_service.py	1	0	0	1
tests/test_google_data_access.py	1	0	0	1
tests/test_gen3_sdk.py	1	0	0	1
tests/test_ras_passport.py	0	0	2	2
TOTAL	40	1	5	46

Test summary after rerunning failed integration tests

filepath	passed	SUBTOTAL
tests/test_graph_submit_and_query.py	1	1
TOTAL	1	1

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

github-actions · 2026-10-07T18:29:51Z

Integration Tests

Test summary after running integration tests

filepath	passed	failed	skipped	SUBTOTAL
tests/test_graph_submit_and_query.py	12	1	1	14
tests/test_data_upload.py	8	0	1	9
tests/test_presigned_url.py	8	0	0	8
tests/test_dbgap.py	2	2	1	5
tests/test_centralized_auth.py	5	0	0	5
tests/test_audit_service.py	1	0	0	1
tests/test_gen3_sdk.py	1	0	0	1
tests/test_ras_passport.py	0	0	2	2
tests/test_google_data_access.py	0	1	0	1
TOTAL	37	4	5	46

Test summary after rerunning failed integration tests

filepath	passed	failed	SUBTOTAL
tests/test_dbgap.py	2	0	2
tests/test_graph_submit_and_query.py	1	0	1
tests/test_google_data_access.py	0	1	1
TOTAL	3	1	4

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

github-actions · 2026-10-07T18:30:05Z

Failure Analysis

Please find the detailed test analysis report here

github-actions · 2026-10-07T19:29:37Z

Integration Tests

Test summary after running integration tests

filepath	passed	failed	skipped	SUBTOTAL
tests/test_graph_submit_and_query.py	12	1	1	14
tests/test_data_upload.py	8	0	1	9
tests/test_presigned_url.py	8	0	0	8
tests/test_centralized_auth.py	5	0	0	5
tests/test_audit_service.py	1	0	0	1
tests/test_dbgap.py	4	0	1	5
tests/test_google_data_access.py	1	0	0	1
tests/test_gen3_sdk.py	1	0	0	1
tests/test_ras_passport.py	0	0	2	2
TOTAL	40	1	5	46

Test summary after rerunning failed integration tests

filepath	passed	SUBTOTAL
tests/test_graph_submit_and_query.py	1	1
TOTAL	1	1

Please find the detailed integration test report here

Please find the detailed integration test report after rerunning failed tests here

Please find the Github Action logs here

mfshao and others added 4 commits September 25, 2026 14:43

make query parameter optional

1323be4

make gen3metadata gracefully handle gen3auth

4586dfe

update version

676446c

Apply automatic documentation changes

ac7d2ea

dummy commit

fd1ad6f

SpencerAxelrod reviewed Oct 5, 2026

View reviewed changes

mfshao and others added 3 commits October 5, 2026 16:03

remove unused parameter

41727be

address for 204

53ccd50

Apply automatic documentation changes

6c4d67a

fix test

c4196d8

BinamB approved these changes Oct 8, 2026

View reviewed changes

mfshao merged commit 1f4d691 into master Oct 9, 2026
18 of 19 checks passed

mfshao deleted the feat/improve-query branch October 9, 2026 14:35

	partial_metadata = mds.query(
	f"_guid_type={guid_type}",
	return_full_metadata=True,
	limit=min(limit, max_guids_per_request),
	offset=offset,
	use_agg_mds=use_agg_mds,

		@@ -588,12 +654,14 @@ def delete_alias(self, guid, alias, **kwargs):
		return response.json()

Conversation

mfshao commented Sep 30, 2026

New Features

Breaking Changes

Bug Fixes

Improvements

Dependency updates

Deployment changes

Uh oh!

github-actions Bot commented Sep 30, 2026

Uh oh!

github-actions Bot commented Sep 30, 2026

Integration Tests

Uh oh!

github-actions Bot commented Sep 30, 2026

Failure Analysis

Uh oh!

github-actions Bot commented Oct 1, 2026

Integration Tests

Uh oh!

SpencerAxelrod Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

mfshao Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

SpencerAxelrod Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

mfshao Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

SpencerAxelrod Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

mfshao Oct 5, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Oct 5, 2026

Integration Tests

Uh oh!

github-actions Bot commented Oct 7, 2026

Integration Tests

Uh oh!

github-actions Bot commented Oct 7, 2026

Failure Analysis

Uh oh!

github-actions Bot commented Oct 7, 2026

Integration Tests

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants