Skip to content

Support for several models and more customizations - #42

Draft
christophe0606 wants to merge 2 commits into
Arm-Examples:mainfrom
christophe0606:multiplemodels
Draft

christophe0606 wants to merge 2 commits into
Arm-Examples:mainfrom
christophe0606:multiplemodels

Conversation

@christophe0606

@christophe0606 christophe0606 commented Oct 23, 2026 •

Copy link
Copy Markdown

Goal of the PR:

  • We may have several methods in the same .pte to support more complex data pipelines
  • We may need metadata in the .pte to have a code that is more generic
  • We may need different .pte for different data pipelines
  • Ethos-U quantization can use different scheme (get_symmetric_quantization_config or get_symmetric_a16w8_quantization_config)
  • We may work with networks that have already been quantized

To support those use cases, we need a more complex contract between create_ai_layer.py and model.py and probably a rework of the model node in mlops.

The new python description.py provides datatype to describe more complex use-cases.
model.py can be used as before and nothing will change. But, by using the new datatype it can describes different needs to create_ai_layer.py.

It introduces new classes : Model, Metadata, Pte

The Model class describes:

  • Different name for the model
  • If the model is already quantized or not and the function to use to provide calibration samples
  • If the model needs a custom quantization (not default one from create_ai_layer) and the function to use to do this custom quantization
  • Information like input-shape and calibration-samples no more read from the mlops

A Metadata class to provide metadata in the pte

A Pte class to package several models + metadata in the pte

A list of Pte object to generate several Pte.

The get_model function can thus generate:

  1. A nn.Module as before
  2. A Model
  3. A list of Model
  4. A Pte containing one or several Model
  5. A list of Pte

The generated ai_layer will contain all operators needed by all networks of all pte.
There will be a different group per pte.

Not covered:

  1. Multiple NPU support
  2. Custom vela options per .pte
  3. Future vela features like VelaExternalBlockPlacements
  4. More customization for memory management

Reference issue : #41

@christophe0606

Copy link
Copy Markdown
Author

I'll probably move some information given through the dataclasses to the mlops description in csolution ... so this PR is a work in progress

@christophe0606
christophe0606 marked this pull request as draft October 23, 2026 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant