Merkleize only hashes physically identical node layouts to the same value

[adamv-symbolica]

So sharing is limited to subtries that were created in the same way, which dramatically limits its applicability because of the non-canonical nature of the heterogeneous node types.

[luketpeterson]

This is actually a really tricky topic. If you recall, Remy's first Merkelizer, which was part of the first serialization module, did an exhaustive Merkelization on the logical trie. It was a disaster. Both for memory consumption and runtime performance.

There are all kinds of degenerate cases. For example, Individual 1-byte path tips ended up in their own nodes... What we really need is a good set of heuristics that are informed by, but not constrained by, the physical representation. The guiding idea being to re-express each subsection of a trie in its most optimal representation, and then Merkelize on those.

Certainly something to strive towards, but it's a somewhat big task. I guess with the clankers' help it's no bigger than many other tasks we've accomplished. But also not something that will be finished in an afternoon.

[adamv-symbolica]

That's not the expectation I'm raising: the boundaries can be physical, but the physical layout should not dictate when they're equivalent. Related https://github.com/adamv-symbolica/PathMap/tree/merkle-cata