[jorenham]

Hi there,

Pyright does consider the from x import x imports in __init__.py as public interface, causing it to report errors when attempting to import from torchcodec or any of the subpackes. For example:

from torchcodec.decoders import VideoDecoder
import torchcodec
torchcodec.decoders.AudioDecoder

Here, pyright reports:

/home/joren/Workspace/torchcodec/repro.py
  /home/joren/Workspace/torchcodec/repro.py:1:33 - error: "VideoDecoder" is not exported from module "torchcodec.decoders"
    Import from "torchcodec.decoders._video_decoder" instead (reportPrivateImportUsage)
  /home/joren/Workspace/torchcodec/repro.py:3:21 - error: "AudioDecoder" is not exported from module "..decoders" (reportPrivateImportUsage)
2 errors, 0 warnings, 0 informations

Simiarly with mypy --strict:

repro.py:1: error: Module "torchcodec.decoders" does not explicitly export attribute "VideoDecoder"  [attr-defined]
repro.py:3: error: Module "torchcodec.decoders" does not explicitly export attribute "AudioDecoder"  [attr-defined]
Found 2 errors in 1 file (checked 1 source file)

Type-checking aside, measuring type coverage also doesn't work right now:

$ uvx pyrefly coverage check --public-only src/torchcodec
 INFO type coverage 100.00% (0 of 0 typable)

(note the "0 of 0 typable")

---

I saw in #1443 that __all__ wasn't a desired solution, which I think is fair. So instead, I went with the import x as x re-export approach, which is officially supported by Python's typing spec, and understood by all major static type-checkers. PyTorch itself also uses this convention: https://github.com/pytorch/pytorch/blob/dd26d1703bfefdd03d1363f50052b5146253e734/torch/nn/__init__.py#L2-L18

With this fix, pyright and mypy no longer report errors for the example above, and pyrefly coverage --public-only now "sees" 291 typables instead of 0.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/meta-pytorch/torchcodec/1786

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[NicolasHug]

Sorry I don't think we'll do this. I still don't get why type checkers try their hardest to invalidate perfectly fine code.

[jorenham]

TBH I also find it strange that type-checkers behave this way. But many people rely on these type-checkers to prevent bugs in their code. Right now, that doesn't work for torchcodec, at all.
So yea, it's not the prettiest, but it'll help a lot of users. And as biased as I might be, I absolutely think this is worth it.

[jorenham]

I still don't get why type checkers try their hardest to invalidate perfectly fine code.

I believe their main argument is that for modules with many imports, you often don't intend to expose those imports as public interface. If type-checkers treat each import as externally visible, then IDE features like import autocompletion would become very prone to importing stuff not meant to be imported, e.g. from torchcodec import os. If you then remove the os import in torchcodec/__init__.py after a while, the result is an unintended breaking change.

And FWIW: torch itself also uses this convention: https://github.com/pytorch/pytorch/blob/dd26d1703bfefdd03d1363f50052b5146253e734/torch/nn/__init__.py#L2-L17