From 12aad676beb451afc24f71126be54979c5f61df8 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Tue, 15 Sep 2026 22:18:11 +0200 Subject: [PATCH 1/5] Improved entrypage for creating configuration. --- docs/source/explanation/configuration.md | 2 +- .../configuration/create-configuration.md | 19 +++++++++++++------ 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index 839e256..d3b04f9 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -1,4 +1,4 @@ -# Principles and Syntax of the Configuration +# Configuration Structure and Syntax By creating a configuration it is possible to have pyAML build devices and applications automatically for several control modes. diff --git a/docs/source/how-to/configuration/create-configuration.md b/docs/source/how-to/configuration/create-configuration.md index 00de849..671290b 100644 --- a/docs/source/how-to/configuration/create-configuration.md +++ b/docs/source/how-to/configuration/create-configuration.md @@ -1,24 +1,31 @@ # Create and Load Configuration -The principles and syntax of the configuration are explained in more detail in [Principles and Syntax of the Configuration](../../explanation/configuration). This guide focuses on the different ways to create it. +The structure and syntax of the configuration are explained in detail in [Configuration Structure and Syntax](../../explanation/configuration). This guide focuses on the different ways to create it. -There are several ways to create a configuration. It is recommended to test the different options and see which one you prefer: +There are many ways to create a configuration. It is recommended to test the different +options and see which one you prefer: -- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects and export as a dictionary or text file +- Use a [JSON Schema in VS Code](./use-vscode-json-schema.md) - Use a JSON Schema in the [MetaConfigurator](./use-meta-configurator.md) -- Use a [JSON Schema in VS Code](./use-vscode-json-schema.md) +- [Use ConfigurationSchema](./use-configuration-schema.ipynb) objects and export as a dictionary or text file + +Another option is to use AI coding assistance tools. You can then for example supply a lattice file, information describing the naming conventions for your control system and a JSON Schema for the pyAML configuration and get help to write it. + +For information about what a JSON Schema is and how to generate it, see [Configuration Schemas and Validation](../../explanation/schema_and_validation.md) and [Generate JSON Schemas](./generate-json-schema.ipynb). ## Load the Configuration -The configuration can be loaded into the `Accelerator` in several ways: +The configuration can be loaded into the `Accelerator` in two ways: | Type| Command | Description | | --- | --- | --- | -| File | `Accelerator.load()` | A text file in JSON on YAML format. +| File | `Accelerator.load()` | A text file in JSON or YAML format. | Dictionary | `Accelerator.from_dict()` | A nested dictionary. +See the API documentation for the [Accelerator](https://pyaml.readthedocs.io/en/stable/api/pyaml.accelerator.html#module-pyaml.accelerator) for more details. + ## Validation The configuration is validated when loading it into the `Accelerator` but it can also be validated without having to load it. This is useful if you want to be able to maintain it separately from pyAML. See [Validate Configuration](./validate-configuration) for details. \ No newline at end of file From ad951aee5323729274ef57eee5cc0e65a23c8cd0 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Tue, 15 Sep 2026 22:36:03 +0200 Subject: [PATCH 2/5] Improve configuration explanation. --- docs/source/explanation/configuration.md | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/docs/source/explanation/configuration.md b/docs/source/explanation/configuration.md index d3b04f9..bd2f7f3 100644 --- a/docs/source/explanation/configuration.md +++ b/docs/source/explanation/configuration.md @@ -45,19 +45,19 @@ model: The configuration describes what should be constructed; it does not contain executable Python code. This keeps configuration readable, reviewable, and usable by tools such as JSON Schema editors. -The configuration is possible to maintain separately to pyAML, for example in a separate Git repository or in a database. +The configuration is possible to maintain separately from pyAML, for example in a separate Git repository or in a database. ## Supported Formats The configuration can be written and loaded in different formats: -**File**: It can be written as a text file and loaded using `Accelerator.load()`. Both `YAML` and `JSON` are supported but `YAML` is considered the default option. +**File**: It can be written as a text file and loaded using `Accelerator.load()`. Both YAML and JSON are supported. **Dictionary**: It can also be written as a nested dictionary and loaded using `Accelerator.from_dict()`. ## Configuration Root -The configuration root is the directory used to resolve relative configuration paths. It applies to the file passed to `Accelerator.load()`, paths used by resolvers, and automatic file includes. Relative paths are resolved against this directory. +The configuration root is the directory used to resolve relative configuration paths. It applies to the file passed to `Accelerator.load()` and paths used by [resolvers](#resolvers. Relative paths are resolved against this directory. By default, the root is the current working directory when pyAML is imported. It can be changed before loading a configuration with `ROOT.set()`: @@ -67,11 +67,15 @@ from pyaml.configuration import ROOT ROOT.set("/path/to/configuration") ``` -After setting the root, `devices/quadrupole.yaml` refers to `/path/to/configuration/devices/quadrupole.yaml`. Absolute paths are normalized and used directly. Setting the root makes it possible to keep a configuration and its included files in a portable directory tree while selecting that tree at runtime. +After setting the root, a file written as `devices/quadrupole.yaml` in the configuration file will refer to `/path/to/configuration/devices/quadrupole.yaml`. + +Setting the root makes it possible to keep a configuration and its included files in a portable directory tree while selecting that tree at runtime. + +Absolute paths are normalized and used directly. ## Resolvers -Resolvers let a configuration value refer to information that is supplied when the configuration is loaded, such as an environment variable or another configuration file. A resolver expression has the following form: +Resolvers can be used in the configuration to let a value refer to information which is supplied when the configuration is loaded, for example an environment variable or another configuration file. A resolver expression has the following form: ```yaml ${resolver:payload} @@ -85,6 +89,9 @@ The following built-in resolvers are available: | --- | --- | --- | | `env` | Environment variable | The value of the named environment variable. An error is raised if it is not set. | | `path` | A file or directory path | The absolute, normalized path, resolved relative to pyAML's configuration root. The target is not loaded as part of loading the configuration. | -| `file` | A YAML, YML, or JSON file path | The path to a file which should be loaded and expanded into the configuration as part of loading the configuration. Relative paths use the configuration root. +| `file` | A YAML, YML, or JSON file path | The path to a file which should be loaded and expanded into the configuration as part of loading it. Relative paths use the configuration root. + +Configuration files can also be included without an explicit `file` expression. A string ending in `.yaml`, `.yml`, or `.json` is loaded automatically for convenience. This makes it easy to split the configuration into several files if one wishes. -Configuration files can also be included without an explicit `file` expression. A string ending in `.yaml`, `.yml`, or `.json` is loaded automatically for convenience if one wishes to split the configuration into several files. +```{warning} +If you include a `.yaml`, `.yml`, or `.json` in the configuration file which you do not want to be loaded and expanded into the configuration (for example a lattice in JSON format), remember to put `${path:filename}` or you will get an error when loading the configuration. From 54219bc2138e4ade8c68e9d0b34eb47ab70bd096 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Tue, 15 Sep 2026 22:56:22 +0200 Subject: [PATCH 3/5] Cleanup of the schema registry guide. --- .../configuration/use-schema-registry.ipynb | 23 +++++++++++++------ 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/docs/source/how-to/configuration/use-schema-registry.ipynb b/docs/source/how-to/configuration/use-schema-registry.ipynb index 9b62b72..009ec95 100644 --- a/docs/source/how-to/configuration/use-schema-registry.ipynb +++ b/docs/source/how-to/configuration/use-schema-registry.ipynb @@ -7,9 +7,9 @@ "source": [ "# Use the Schema Registry\n", "\n", - "This guide shows how to register configuration schemas in the `SchemaRegistry` and how to explore the contents of the registry.\n", + "This guide shows how to register configuration schemas in the `SchemaRegistry` and how to explore the contents of the registry. The full functionality is available in the [API documentation](https://pyaml.readthedocs.io/en/stable/api/pyaml.validation.html#pyaml.validation.SchemaRegistry).\n", "\n", - "The registry can be used to [validate configuration](./validate-configuration) or [generate JSON Schema](./generate-json-schema)." + "The registry can be used to [validate configuration](./validate-configuration) or [generate JSON Schemas](./generate-json-schema). More explanation can be found in [Configuration Schemas and Validation](./../../explanation/schema_and_validation.md)." ] }, { @@ -65,7 +65,7 @@ "\n", " The decorator can be used in two ways: dynamically generating the schema from the class constructor or by explicitly declaring a schema too use for the class.\n", "\n", - " Note: a decorator only runs when a module is imported. If you use the `register_schema` decorator in a module that is not imported the schema will not be registered.\n", + " > Note: A decorator only runs when a module is imported. If you use the `register_schema` decorator in a module that is not imported the schema will not be registered.\n", "\n", "The different ways to use the decorator is explained below." ] @@ -102,7 +102,7 @@ "source": [ "The schema is now visible in the registry. In this case the schema will get the path `__main__.Magnet` since the class was declared directly in the script. For other classes it will be of the form `package.module.Class`.\n", "\n", - "You can use `describe` to get pretty output of the fields in the schema. `ConfigurationSchema` inherits from Pydantic `BaseModel` so the functionality of that class is also available. See Pydantic's documentation for details." + "You can use `describe` to get output of the fields in the schema. `ConfigurationSchema` inherits from [Pydantic BaseModel](https://pydantic.dev/docs/validation/latest/concepts/models/) so the functionality of that class is also available. See the [Pydantic documentation](https://pydantic.dev/docs/validation/latest/get-started/) for details." ] }, { @@ -140,7 +140,7 @@ "source": [ "### Register an Explicit Schema\n", "\n", - "Define a `ConfigurationSchema` explicitly when you need more control over the configuration fields or validation rules. The functionality of Pydantic is available when defining the schema.\n", + "Define a `ConfigurationSchema` explicitly when you need more control over the configuration fields or validation rules. The functionality of `Pydantic` is available when defining the schema.\n", "\n", "The explicit schema must inherit from `ConfigurationSchema` since that defined the minimum required fields for all items in the registry." ] @@ -199,7 +199,16 @@ "source": [ "## Automatically Discover Schemas\n", "\n", - "Since registration only happens when the modules containing decorated classes are imported, a `discover` method is available to automatically scan packages and register the schemas in the package." + "Since registration only happens when the modules containing decorated classes are imported, a `discover` method is available to automatically scan packages and register schemas in the package.\n", + "\n", + "The method automatically scans the `pyaml` package. For other packages, they need to define entry points in their `pyproject.toml` for the schemas in those packages to be discovered. For example:\n", + "\n", + "```toml\n", + "[project.entry-points.\"pyaml.schemas\"]\n", + "pyaml_cs_oa = \"pyaml_cs_oa\"\n", + "```\n", + "\n", + "The entry point supports both defining a whole package or individual modules. For packages the `discovery` will recursively import schemas in all modules in the package." ] }, { @@ -313,7 +322,7 @@ "source": [ "## Browse the Registry\n", "\n", - "The registry also supports common mapping operations. Class paths are the keys and registered schema classes are the values. See the API documentation for all available methods." + "The registry also supports common mapping operations. Class paths are the keys and registered schema classes are the values. See the [API documentation](https://pyaml.readthedocs.io/en/stable/api/pyaml.validation.html#pyaml.validation.SchemaRegistry) for all available methods." ] }, { From 8a858ba8f267a5845ea141c7508cae94ebdd7b50 Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Tue, 15 Sep 2026 23:08:01 +0200 Subject: [PATCH 4/5] Updated validate guide. --- .../validate-configuration.ipynb | 24 ++++++++++++++----- 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/docs/source/how-to/configuration/validate-configuration.ipynb b/docs/source/how-to/configuration/validate-configuration.ipynb index 593f2f1..06cacfd 100644 --- a/docs/source/how-to/configuration/validate-configuration.ipynb +++ b/docs/source/how-to/configuration/validate-configuration.ipynb @@ -7,9 +7,13 @@ "source": [ "# Validate Configuration\n", "\n", - "The configuration can be validated using the `SchemaValidator`. It makes use of the schema registry to extract which schema to validate against for a specific class.\n", + "To make sure that the input data provided to pyAML has the correct format, validation is important. The validation is by default automatically performed when loading a configuration into the `Accelerator`, but it can also be done separately if one wishes. This guide shows how to do that.\n", "\n", - "For validation to be possible the class must be registered in the schema registry. If the class is not registered, validation will be skipped, a warning given and the data kept unchanged. Beware that this can lead to unexpected errors." + "For more details about schemas and the different types of validation, see [Configuration Schemas and Validation](./../../explanation/schema_and_validation.md).\n", + "\n", + "The configuration can be validated using the [SchemaValidator](https://pyaml.readthedocs.io/en/stable/api/pyaml.validation.html#pyaml.validation.SchemaValidator). It makes use of the schema registry to extract which schema to validate against for a specific class.\n", + "\n", + "For validation to be possible the class must be registered in the schema registry." ] }, { @@ -21,7 +25,7 @@ "\n", "First create the registry and register the classes you want included in the schema.\n", "\n", - "Here the discovery function is used to register all classes in `pyaml` and other packages which define entry points." + "Here the discovery function is used to register all classes in `pyaml` and other packages which define entry points. See [Use the Schema Registry](./use-schema-registry) for details." ] }, { @@ -47,19 +51,25 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "id": "b2523c34", "metadata": {}, "outputs": [], "source": [ "from pyaml.validation import SchemaValidator\n", "\n", + "# Create some configuration\n", "configuration = {\n", " \"class_path\": \"pyaml.magnet.quadrupole.Quadrupole\",\n", " \"name\": \"QF1\",\n", " \"description\": \"This is the QF1 quadrupole magnet.\"\n", "}\n", - "validated = SchemaValidator.validate(configuration)" + "\n", + "# Validate it\n", + "validated = SchemaValidator.validate(configuration)\n", + "\n", + "# Print the validated ConfigurationSchema object\n", + "print(validated)" ] }, { @@ -72,7 +82,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "id": "2e5df599", "metadata": {}, "outputs": [ @@ -97,6 +107,8 @@ "}\n", "\n", "validated = SchemaValidator.validate(configuration)\n", + "\n", + "# Print the validated ConfigurationSchema object\n", "print(validated)" ] }, From ea30760555222ed3b7be362545079f1bed0df69f Mon Sep 17 00:00:00 2001 From: Teresia Olsson Date: Tue, 15 Sep 2026 23:19:54 +0200 Subject: [PATCH 5/5] Cleanup generate json schema. --- .../configuration/generate-json-schema.ipynb | 26 +++++++++---------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/source/how-to/configuration/generate-json-schema.ipynb b/docs/source/how-to/configuration/generate-json-schema.ipynb index 3e74178..ba5e34e 100644 --- a/docs/source/how-to/configuration/generate-json-schema.ipynb +++ b/docs/source/how-to/configuration/generate-json-schema.ipynb @@ -7,7 +7,7 @@ "source": [ "# Generate JSON Schemas\n", "\n", - "This guide shows how to use the `SchemaRegistry` to generate JSON Schemas." + "This guide shows how to use the `SchemaRegistry` to generate [JSON Schemas](https://json-schema.org/). For details about what a JSON Schema and how it can be used, see [Configuration Schemas and Validation](./../../explanation/schema_and_validation.md)." ] }, { @@ -19,7 +19,7 @@ "\n", "First create the registry and register the classes you want included in the schema.\n", "\n", - "Here the discovery function is used to register all classes in `pyaml` and other packages which define entry points." + "Here the discovery function is used to register all classes in `pyaml` and other packages which define entry points. See [Use the Schema Registry](./use-schema-registry) for details." ] }, { @@ -42,14 +42,14 @@ "source": [ "## Generate JSON Schema\n", "\n", - "Use the `SchemaGenerator` to generate a JSON Schema to use with external tools.\n", + "Use the [SchemaGenerator](https://pyaml.readthedocs.io/en/stable/api/pyaml.validation.html#pyaml.validation.SchemaGenerator) to generate a JSON Schema to use with external tools.\n", "\n", "If a base schema has registered concrete or virtual subclasses, the generated JSON Schema includes those alternatives. This allows editors and other JSON Schema tools to offer the appropriate fields for each configuration type." ] }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "id": "dfb96bb7", "metadata": {}, "outputs": [ @@ -68,7 +68,7 @@ " 'lattice_names': {'default': None,\n", " 'title': 'Lattice Names',\n", " 'type': ['string', 'null']},\n", - " 'model': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'model': {'anyOf': [{'oneOf': [{'additionalProperties': False,\n", " 'properties': {'class': {'const': 'pyaml.magnet.identity_cfm_model.IdentityCFMagnetModel',\n", " 'description': 'Fully '\n", " 'qualified '\n", @@ -143,7 +143,7 @@ " 'path.',\n", " 'title': 'Class',\n", " 'type': 'string'},\n", - " 'curves': {'items': {'anyOf': [{'additionalProperties': False,\n", + " 'curves': {'items': {'oneOf': [{'additionalProperties': False,\n", " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", " 'description': 'Fully '\n", " 'qualified '\n", @@ -180,7 +180,7 @@ " 'title': 'Hardware '\n", " 'Units',\n", " 'type': 'array'},\n", - " 'matrix': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'matrix': {'anyOf': [{'oneOf': [{'additionalProperties': False,\n", " 'properties': {'class': {'const': 'pyaml.magnet.csvmatrix.CSVMatrix',\n", " 'description': 'Fully '\n", " 'qualified '\n", @@ -263,7 +263,7 @@ " 'crosstalk': {'default': 1.0,\n", " 'title': 'Crosstalk',\n", " 'type': 'number'},\n", - " 'curve': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'curve': {'anyOf': [{'oneOf': [{'additionalProperties': False,\n", " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", " 'description': 'Fully '\n", " 'qualified '\n", @@ -337,7 +337,7 @@ " 'type': 'array'}],\n", " 'default': 1.0,\n", " 'title': 'Crosstalk'},\n", - " 'curves': {'anyOf': [{'anyOf': [{'additionalProperties': False,\n", + " 'curves': {'anyOf': [{'oneOf': [{'additionalProperties': False,\n", " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", " 'description': 'Fully '\n", " 'qualified '\n", @@ -368,7 +368,7 @@ " 'title': 'InlineCurveConfigurationSchema',\n", " 'type': 'object'}],\n", " 'title': 'CurveConfigurationSchema'},\n", - " {'items': {'anyOf': [{'additionalProperties': False,\n", + " {'items': {'oneOf': [{'additionalProperties': False,\n", " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", " 'description': 'Fully '\n", " 'qualified '\n", @@ -439,7 +439,7 @@ " 'crosstalk': {'default': 1.0,\n", " 'title': 'Crosstalk',\n", " 'type': 'number'},\n", - " 'curve': {'anyOf': [{'additionalProperties': False,\n", + " 'curve': {'oneOf': [{'additionalProperties': False,\n", " 'properties': {'class': {'const': 'pyaml.magnet.csvcurve.CSVCurve',\n", " 'description': 'Fully '\n", " 'qualified '\n", @@ -498,9 +498,9 @@ ], "source": [ "from pprint import pprint\n", - "\n", "from pyaml.validation import SchemaGenerator\n", "\n", + "# Generate the JSON Schema for a quadrupole magnet\n", "json_schema = SchemaGenerator.generate(\"pyaml.magnet.quadrupole.Quadrupole\")\n", "pprint(json_schema)" ] @@ -510,7 +510,7 @@ "id": "d329d7e5", "metadata": {}, "source": [ - "The result can also be saved directly to a file." + "The result can also be saved directly to a file. As can be seen, the schema is very long and not easy to read but it is not meant to be used by humans, only by tools which make use of the JSON Schema standard." ] }, {