Add dataclass encoding by zjkmxy · Pull Request #108 · named-data/python-ndn

zjkmxy · 2026-08-17T04:37:50Z

Ref: #97

New wire format is longer but with advantage:

Built-in dataclass support, such as =, repr and asdict. Also, models can be easily encoded into JSON and other formats for human/subagents to read.
Built-in typing annotation. Type checker now can correctly understand field types.
Faster than v1. Using type hints cache, v2 is about 80% of the time in v1 for encoding and decoding.

Not so good points:

v2 typing is longer than v1.
Signature representation is still ugly.
Annotation handling could change with interpreters.
bytes field could actually hold memoryview. Need further effort to improve.

I plan to replace old model with the new encoding model in a future breaking change. This diff adds new method as extra files for people to review.

Co-authored-by: Cursor <cursoragent@cursor.com>

Copilot

Copilot review overview

🟡 Changes recommended

The new codec and application stack contain unresolved decoding, buffer sizing, reply-contract, and LP framing bugs.

Review effort: Balanced
Findings: 2 · 5

Open (7)

Include fragment payload length in LP packet length · New
Validate fixed_len before sizing and encoding · New
Handle missing fragments before parsing fragment type · New
Return True after successfully sending the response · New
Validate nested value against annotated model type · New
Reject TLV values exceeding the remaining buffer · New
Validate map value TLV type and bounds · New

What changed in this PR

Adds a parallel dataclass-based TLV encoding stack and migrates selected applications, commands, and examples to it.

Changes:

Implements dataclass TLV encoding, parsing, maps, signatures, and schema caching.
Adds dataclass packet, NDNLP, security, NFD management, and application modules.
Migrates CLI tools/examples and adds compatibility and integration tests.

File	Description
`tests/misc/security_v2_2_test.py`	Tests dataclass certificate and SafeBag support.
`tests/misc/nfd_mgmt_2_test.py`	Tests NFD management interoperability.
`tests/integration/app_v2_2_test.py`	Exercises the dataclass-based application stack.
`tests/encoding/tlv_model_v2_test.py`	Tests the new TLV codec.
`tests/encoding/ndnlp_v2_2_test.py`	Tests dataclass NDNLP handling.
`tests/encoding/ndn_format_0_3_2_test.py`	Tests packet encoding and signatures.
`src/ndn/transport/nfd_registerer.py`	Adds dataclass management registration.
`src/ndn/security/tpm/tpm_osx_keychain.py`	Formats key names in errors.
`src/ndn/encoding/tlv_model_v2.py`	Implements dataclass TLV serialization.
`src/ndn/encoding/ndnlp_v2_2.py`	Adds dataclass NDNLP models.
`src/ndn/encoding/ndn_format_0_3_2.py`	Adds dataclass packet models.
`src/ndn/encoding/__init__.py`	Exports the new TLV API.
`src/ndn/bin/sec/utils.py`	Uses dataclass security constants.
`src/ndn/bin/sec/cmd_sign_cert.py`	Migrates certificate signing.
`src/ndn/bin/sec/cmd_import_cert.py`	Migrates certificate importing.
`src/ndn/bin/sec/cmd_get_signreq.py`	Migrates signing requests.
`src/ndn/bin/sec/cmd_get_childitem.py`	Migrates certificate display.
`src/ndn/bin/nfdc/utils.py`	Uses the new application stack.
`src/ndn/bin/nfdc/cmd_set_strategy.py`	Migrates strategy commands.
`src/ndn/bin/nfdc/cmd_remove_strategy.py`	Migrates strategy removal.
`src/ndn/bin/nfdc/cmd_remove_route.py`	Migrates route removal.
`src/ndn/bin/nfdc/cmd_remove_face.py`	Migrates face removal.
`src/ndn/bin/nfdc/cmd_new_route.py`	Migrates route creation.
`src/ndn/bin/nfdc/cmd_new_face.py`	Migrates face creation.
`src/ndn/bin/nfdc/cmd_get_strategy.py`	Migrates strategy queries.
`src/ndn/bin/nfdc/cmd_get_status.py`	Migrates status queries.
`src/ndn/bin/nfdc/cmd_get_route.py`	Migrates route queries.
`src/ndn/bin/nfdc/cmd_get_face.py`	Migrates face queries.
`src/ndn/appv2_2.py`	Adds the dataclass-based application implementation.
`src/ndn/app_support/security_v2_2.py`	Adds dataclass security models.
`src/ndn/app_support/nfd_mgmt_2.py`	Adds dataclass NFD management models.
`src/ndn/app_support/light_versec/checker.py`	Uses dataclass certificate parsing.
`src/ndn/app_support/light_versec/binary.py`	Converts LVS models to dataclasses.
`examples/dpdk_experimental/udp_producer.py`	Migrates the DPDK producer.
`examples/dpdk_experimental/udp_consumer.py`	Migrates the DPDK consumer.
`examples/appv2/keychain_cert/keychain_register.py`	Migrates keychain registration.
`examples/appv2/keychain_cert/fetch_certificate.py`	Migrates certificate fetching.
`examples/appv2/forwarding_hint/producer.py`	Migrates the forwarding-hint producer.
`examples/appv2/forwarding_hint/consumer.py`	Migrates the forwarding-hint consumer.
`examples/appv2/basic_packets/producer.py`	Migrates the basic producer.
`examples/appv2/basic_packets/consumer.py`	Migrates the basic consumer.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot

Copilot review overview

🟡 Changes recommended

Schema resolution, Interest validation, MetaInfo compatibility, and premature LVS API replacement need correction.

Review effort: Balanced
Findings: 1

Open (1)

Convert v1 MetaInfo to v2 before nested-model encoding · New

Resolved since last review (7)

Validate fixed_len before sizing and encoding
Include fragment payload length in LP packet length
Validate map value TLV type and bounds
Reject TLV values exceeding the remaining buffer
Validate nested value against annotated model type
Return True after successfully sending the response
Handle missing fragments before parsing fragment type

Previously missed (3)

In code that hasn't changed since last review

Preserve compatibility with exported LVS model classes

src/ndn/app_support/light_versec/binary.py:70

This replaces the existing exported LVS model classes in place and removes inherited public methods such as encode, parse, and asdict from every class except LvsModel. That is a breaking API change now, despite the PR description saying replacement is deferred; keep binary.py compatible and introduce the dataclass models side-by-side (or provide compatibility methods for all exported classes).

Resolve local forward references when building dataclass schemas

src/ndn/encoding/tlv_model_v2.py:275

Locally declared self-referential dataclasses cannot build a schema: get_type_hints(cls) resolves the forward reference only from module globals, so a field such as child: 'Node | None' raises NameError on the first encode/parse. Supply the class itself as a local namespace (and add a regression test combining local classes with forward references).

Reject digest components when ApplicationParameters are absent

src/ndn/encoding/tlv_model_v2.py:332

This now accepts a ParametersSha256DigestComponent when ##need_digest is false. The v1 encoder rejects that component unless ApplicationParameters are present, because otherwise the Interest carries an unnecessary parameters digest; restore that validation to retain packet-format behavior.

yoursunny

These v2_2 filenames are very confusing.
Just do the breaking change now, so that I can clearly see the diff.

yoursunny · 2026-10-07T21:22:21Z

+# -----------------------------------------------------------------------------
+"""NDN Packet Format v0.3 models using the dataclass TLV API."""
+import dataclasses as dc
+from typing import Optional


Minimum is Python 3.11, so that Optional[T] should be written as T | None.
See #109 on how to modernize.

zjkmxy · 2026-10-07T21:26:06Z

These v2_2 filenames are very confusing. Just do the breaking change now, so that I can clearly see the diff.

Makes sense. Let me publish current master and do breaking changes.

zjkmxy and others added 4 commits September 6, 2026 17:58

encoding: Add dataclass TLV model

4b0636b

Co-authored-by: Cursor <cursoragent@cursor.com>

encoding: Add dataclass packet format

85a53cf

Co-authored-by: Cursor <cursoragent@cursor.com>

encoding: Add dataclass NDNLPv2 model

9cdee7e

Co-authored-by: Cursor <cursoragent@cursor.com>

encoding: Add dataclass security model

4cc2e89

Co-authored-by: Cursor <cursoragent@cursor.com>

zjkmxy force-pushed the dev2 branch from 9521b8a to 4cc2e89 Compare September 7, 2026 01:02

zjkmxy added 3 commits September 6, 2026 18:31

fix: Fix compilation error in pypy3

96d1497

feat: Convert app_support to new model and improve performance

4df603d

Migrate examples and CLIs into new model

82d67ba

zjkmxy requested a balanced review from Copilot September 27, 2026 19:59

Copilot started reviewing on behalf of zjkmxy September 27, 2026 19:59 View session

Copilot AI reviewed Sep 27, 2026

View reviewed changes

zjkmxy marked this pull request as ready for review September 27, 2026 20:12

Fix review comments

6d1314c

zjkmxy requested a balanced review from Copilot September 29, 2026 06:38

Copilot started reviewing on behalf of zjkmxy September 29, 2026 06:38 View session

Copilot AI reviewed Sep 29, 2026

View reviewed changes

Comment thread src/ndn/appv2_2.py

yoursunny suggested changes Oct 7, 2026

View reviewed changes

zjkmxy closed this Oct 7, 2026

zjkmxy mentioned this pull request Oct 8, 2026

Add dataclass encoding #111

Open

Navigation Menu

Repository navigation

Add dataclass encoding - #108

Add dataclass encoding#108
zjkmxy wants to merge 8 commits into
named-data:masterfrom
zjkmxy:dev2

zjkmxy commented Aug 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

yoursunny left a comment

Uh oh!

yoursunny Oct 7, 2026

Uh oh!

zjkmxy commented Oct 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

zjkmxy commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

yoursunny left a comment

Choose a reason for hiding this comment

Uh oh!

yoursunny Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

zjkmxy commented Oct 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

zjkmxy commented Aug 17, 2026 •

edited

Loading