Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- New `parameters` field under `processing:expression` to provide a mapping of named input values employed.
- Asset roles `processing-input`, `processing-output`, `processing-parameters` and `processing-results`.
- Recommendation to employ `profile` in Asset using the above roles and examples for common applications.

### Changed

### Deprecated
Expand Down
65 changes: 53 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,18 +160,59 @@ This list is not exhaustive and can be extended with the processing level specif

### Expression Object

| Field Name | Type | Description |
| ---------- | ------ | ----------- |
| format | string | **REQUIRED** The type of the expression that is specified in the `expression` property. |
| expression | \* | **REQUIRED** An expression compliant with the `format` specified. The expression can be any data type and depends on the `format` given, e.g. string or object. |
| Field Name | Type | Description |
| ---------- | --------------- | ----------- |
| format | string | **REQUIRED** The type of the expression that is specified in the `expression` property (see [Format](#format) examples). |
| expression | \* | **REQUIRED** An expression compliant with the `format` specified. The expression can be any data type and depends on the `format` given, e.g. string or object. |
| parameters | Map<string, \*> | Optional parameters relevant for the context of the `format` specified (e.g., _openEO_ or _OGC API - Processes_ execution inputs, a variable name/file mapping for `gdal-calc`, etc. |

#### Format

Potential expression formats with examples:

| Format | Type | Description | Example |
| ----------- | ------ | ----------- | ------- |
| `gdal-calc` | string | A [`gdal_calc.py`](https://gdal.org/programs/gdal_calc.html) expression based on numpy syntax. | `A*logical_or(A<=177,A>=185)` |
| `openeo` | object | [openEO process](https://openeo.org/documentation/1.0/developers/api/reference.html#section/Processes) | [Example](https://raw.githubusercontent.com/Open-EO/openeo-processes/1.2.0/normalized_difference.json) |
| `rio-calc` | string | A [rio-calc](https://rasterio.readthedocs.io/en/latest/topics/calc.html) (RasterIO) expression | `(b4-b1)/(b4+b1)` |
| Format | Type | Description | Example |
| ------------------ | ------ | ----------- | ------- |
| `gdal-calc` | string | A [`gdal_calc.py`](https://gdal.org/programs/gdal_calc.html) expression based on numpy syntax. | `A*logical_or(A<=177,A>=185)` |
| `openeo` | object | An [openEO process](https://openeo.org/documentation/1.0/developers/api/reference.html#section/Processes) | [Example](https://raw.githubusercontent.com/Open-EO/openeo-processes/1.2.0/normalized_difference.json) |
| `ogcapi-processes` | object | An [OGC ProcessDescription](https://docs.ogc.org/is/18-062r2/18-062r2.html#ogc_process_description) | [Example](https://schemas.opengis.net/ogcapi/processes/part1/1.0/examples/json/ProcessDescription.json) |
| `rio-calc` | string | A [rio-calc](https://rasterio.readthedocs.io/en/latest/topics/calc.html) (RasterIO) expression | `(b4-b1)/(b4+b1)` |

> [!NOTE]
> To ensure portability, `format` may employ a specific profile URI from a registered naming authority
> that explicitly describes the represented `expression` without ambiguity.

#### Parameters

The `parameters` object represents a mapping of named entities that can be interpreted by the `format` expression and
their corresponding values. The specific structure of those values and how they can be applied is specific to the `format`.

For example, an _OGC API - Processes_ or _openEO_ taking an input named `threshold` could supply their respective
_ProcessDecription_ or _Process Graph_ in `expression`, and along it, the `"parameters": {"threshold": 0.5}` inputs
it employed for its execution.

## Asset Roles

In certain cases, such as when large data is employed or produced by a processing expression, it may not be convenient
or reasonable to provide inputs and results inline of the STAC object. For this purpose, the following roles are defined
to annotate [Asset Objects](https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md#assets).

| Role | Description |
| --------------------- | ----------- |
| processing-input | An Asset that represents an individual input (e.g., a configuration file required by the processing expression). Note that an input STAC object should instead use the `derived_from` link [relation type](#relation-types). |
| processing-parameters | An Asset that embeds the entire processing parametrization contents that were submitted. This is similar to [Parameters](#parameters) but is not limited to `application/json` and can contain much more data. |
| processing-output | An Asset that represents an invidivudal output (possibily amongst multiple) that was generated by the processing expression. These Assets should typically include `data` role as well, although not strictly required. |
| processing-results | An Asset that embeds the entire processing results contents generated by the processing expression (e.g., an _OGC API - Processes_ [Results](https://schemas.opengis.net/ogcapi/processes/part1/1.0/examples/json/Result.json) document. |

> [!NOTE]
> For each Asset that includes those roles, it is *STRONGLY RECOMMENDED* to include both a `profile` URI and a `type` field
> that will respectively describe the nature of the their contents. Alternate representations of those Assets could then
> be provided using other equivalent content negotiation by profile mechanisms.

> [!WARNING]
> The abscence of the `processing-output` role does **NOT** necessarily indicate that a certain Asset is not a result
> of the processing. This is to support previous versions of the extension and existing STAC definitions that did not
> supply it. Instead, it should be interpreted as *additional indication* to distinguish various Assets amongst
> themselves, notably when inputs and outputs are combined.

## Relation types

Expand All @@ -183,7 +224,7 @@ The following types should be used as applicable `rel` types in the
| derived_from | URL to a STAC Item that was used as input data in the creation of this Item. |
| source_for | URL to a STAC Item that was derived from this Item (forward provenance, inverse of `derived_from`). |
| processing-expression | A processing chain (or script) that describes how the data has been processed. |
| processing-execution | URL to any resource representing the processing execution (e.g. OGC Process API). |
| processing-execution | URL to any resource representing the processing execution (e.g. *OGC API - Processes*). |
| processing-software | URL to any resource that identifies the software and versions used for processing the data, e.g. a `Pipfile.lock` (Python) or `package-lock.json` (NodeJS). |
| processing-validation | URL to any kind of validation that has been applied after processing, e.g. a validation report or a script used for validation. |

Expand All @@ -197,10 +238,10 @@ for running tests are copied here for convenience.

### Running tests

The same checks that run as checks on PR's are part of the repository and can be run locally to verify that changes are valid.
The same checks that run as checks on PR's are part of the repository and can be run locally to verify that changes are valid.
To run tests locally, you'll need `npm`, which is a standard part of any [node.js installation](https://nodejs.org/en/download/).

First you'll need to install everything with npm once. Just navigate to the root of this repository and on
First you'll need to install everything with npm once. Just navigate to the root of this repository and on
your command line run:
```bash
npm install
Expand Down