Repository navigation
fix(conda): vendored dependencies ship without their data files #1814
Description
Describe the bug
The conda package's vendored dependencies (pcapkit/_extern, installed by conda/pypcapkit/build.sh and bld.bat from conda/requirements.txt) reach the wheel only as .py modules, through package discovery. Their data files never ship: for example certifi/cacert.pem, pyshark/config.ini, emoji's JSON data, lxml's compiled extensions and every dist-info. This was found by emulating the recipe (pip install --target of the requirements, then pip wheel .), before and after #1813. A real conda build was not available to confirm it.
If the emulation holds, vendored requests has no CA bundle, and pyshark and emoji lack files they read at runtime.
Expected behavior
A conda-built pcapkit ships each vendored dependency complete, and a smoke test in the conda workflow imports and exercises them.
Additional context
Found by #1794's worker and confirmed by its Sonnet review, which also showed that recursive-include pcapkit/_extern * would add the missing files but warns in every non-conda build. The new MANIFEST.in guard from #1813 would need to allow an include whose target is git-ignored. Fix routes: MANIFEST.in, package-data, or the recipe scripts.
Activity
JarryShaw commentedon Oct 16, 2026
Blocked until #1813 merges; it rewrites
MANIFEST.inand adds the guard any fix here has to satisfy.JarryShaw commentedon Oct 16, 2026
We should probably review what're the needed 3p vendored dependencies for Conda distros. I suppose that list is stale.
JarryShaw commentedon Oct 16, 2026
Unblocked: #1813 has merged. Scope widened, per your note: first audit which third-party packages a conda build actually needs vendored, since
conda/requirements.txtis likely stale againstpyproject.toml, and drop or update the stale ones. Then make the ones that stay ship complete, data files included. A worker is on it. No conda toolchain is available here, so the build will be emulated.JarryShaw commentedon Oct 16, 2026
And also, might wanna automate the audit/update of the list - as the project grows and its dependencies update.
JarryShaw commentedon Oct 16, 2026
Added to the worker's scope: a
util/script that derives the vendored list frompyproject.tomland rewritesconda/requirements.txt, plus a check that fails when the file goes stale. The check will be wired into the conda workflow, or into a scheduled updater like the python-versions one from #1660. All of it lands in the same PR.JarryShaw commentedon Oct 16, 2026
#1821 is up, and it needs one call from you on what the conda package vendors.
dependenciesplus theallextra, matchingpip install pypcapkit[all]since design: write down what belongs in theallextra, and why #910.pcapkit-vendorcrawlers' requests and bs4..tar.bz2is 15.9 MB. pycrate's ASN.1 data is 238 MB unpacked.pip install pycrate.I recommend A. Switching later is one line (
EXTRASinutil/conda_vendor.py).Separately, the macOS
conda convertstep ships arm64 binaries for osx-64. This already broke lxml, and with A it also affects cffi and cryptography. I'd file that as its own issue.JarryShaw commentedon Oct 16, 2026
take A
JarryShaw commentedon Oct 16, 2026
Recorded: option A, which is what #1821 already does:
dependenciesplus theallextra. The osx-64 binary problem is now #1822, blocked behind #1821.fix(conda): vendor only the needed dependencies and ship their data f…
fix(conda): build each macOS conda package on its own architecture (#…
fix(conda): build each macOS conda package on its own architecture (#…
fix(conda): pin cryptography 48.0.1 on Intel macOS, which has no newe…
ci(conda): move platform pins up and give them a review-by date (#1851)