PT: add the 2020, 2021 and 2022 editions - #228
Open
haithcoatj wants to merge 1 commit into
Open
Conversation
These editions publish the sub-parcel geometry as eight or nine regional files in four projections, so each is reprojected to WGS 84 before they are merged; concatenating them as they ship would silently mix coordinate systems. The crop code arrives differently in each: C1 on the layer in 2022, and joined in from a separate table in 2020 and 2021 (left join, validate="many_to_one", row counts asserted per region; maximum multiplicity 1, so fan-out is structurally impossible). Culturas_2021.dbf has fourteen rows keyed Osa_id = 0 with every crop column NULL and no matching geometry anywhere in the edition, which makes the key non-unique; they are dropped, changing no output rows. The layer filter is chosen per edition rather than by the shared DATA_LAYER pattern, because Culturas_.+ matches Culturas_2021 — an attribute-only table with no geometry. None of these editions publishes CUL_ID: IFAP recorded one row per land occupation then, with crops as attributes or in a joined table, so there is no crop parcel to identify. The occupation identifies the field and its parcel is the block, which is the mapping still on main. 2020's crop table stops at the mainland, so its 199,123 island fields carry no crop code. 2021 is unaffected.
m-mohr
requested changes
Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds pt 2020, 2021 and 2022. The commit message has the per-edition mechanics.
This targets publish-portolan and can't be cherry-picked to main the way #227 was. pt.py genuinely differs between the branches: main is blob 25b0769e with no 2025 variant, "OSA_ID": "id" and no DATA_LAYER, while publish-portolan is b78b7b75 with all three. bb2e0bc and 7f33180 both report diverged against main. So CI doesn't run here and everything below was run locally.
Reused from the 2025 work rather than reinvented: use_variant_as_determination, the migrate() rename, the float to int64 cast, and the 6933 recompute. The recompute also gives 2020/2021 metrics:perimeter, so area_calculate_missing is gone. Applying it to 2022, which does ship a provider Shape_Area, costs 0.02% in aggregate and 0.05% in the worst region across 119,400 sampled fields, so it runs unguarded.
Rows 4,766,789 / 4,882,314 / 4,953,834, all four regions with per-region counts summing to the total, uniform EPSG:4326, no null or invalid geometries, schema identical to published 2025.
Broke each mechanism deliberately to confirm a test catches it:
int_cast was removable with the suite green at first, because the delivery assertion checked presence and not format, so id quietly became '2315738.0'. Added an id-format check.
Fixtures are 187-266 KB, in line with the existing ones. One deviation: plain deflate where the real archives are Deflate64, which python zipfile refuses, so the Deflate64 path isn't covered.
crop:name is declared but not produced for these editions. Published 2025 doesn't have it either.
Suite: 43 passed, 4 pre-existing failures (be_wal, ie, si, de_by_block).