Repository navigation
Fix encoding declarations - #325
Merged
Merged
Conversation
Merging this PR will not alter performance
Comparing 
Footnotes
|
There was a problem hiding this comment.
🟡 Changes recommended
Latin-1 alias handling remains incomplete, and BOM-prefixed UTF-8-sig files are not handled correctly.
3 open findings
What changed in this PR
Improves Python source encoding handling in the Rust scanner and propagates decoding failures safely to Python.
Changes:
- Normalize supported encoding declarations and preserve true Latin-1 semantics.
- Replace decoding panics with Python
UnicodeErrorpropagation. - Add functional tests and release metadata.
| File | Description |
|---|---|
rust/src/filesystem.rs |
Adds encoding normalization and structured read errors. |
rust/src/errors.rs |
Maps filesystem errors to Python exceptions. |
rust/src/import_scanning.rs |
Propagates read failures instead of unwrapping. |
tests/functional/test_encoding_handling.py |
Tests aliases, Latin-1, and decoding failures. |
CHANGELOG.rst |
Documents the encoding fix. |
AUTHORS.rst |
Adds the contributor. |
uv.lock |
Synchronizes the project version. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+45
to
+47
| if ["latin-1", "iso-8859-1", "iso-latin-1"] | ||
| .into_iter() | ||
| .any(is_spelling_of) |
Comment on lines
+51
to
+52
| if is_spelling_of("utf-8") { | ||
| return encoding_rs::Encoding::for_label(b"utf-8").map(Encoding::Other); |
Comment on lines
+10
to
+11
| * Fix panic when a module declares its encoding using a name that Python accepts but that isn't a | ||
| WHATWG label, such as ``latin-1`` or ``utf_8``. |
Modules can declare their encoding (PEP 263) using any name that Python
accepts, such as `latin-1`, `utf_8` or `euc_jp`. But encoding_rs only knows
the WHATWG labels (`latin1`, `utf-8`, `euc-jp`...), so these names weren't
recognized, and building the graph panicked with:
pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err`
value: PyErr { type: <class 'TypeError'>, value: TypeError('function
takes exactly 5 arguments (1 given)'), traceback: None }
Normalize the declared name like Python does before looking it up.
Modules that really can't be decoded also caused this panic, as the error
was created as a UnicodeDecodeError, which can't be created from just a
message, and was then unwrapped. Raise it as a UnicodeError instead, which
keeps the message naming the file.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN
The previous commit stored file read errors in GrimpError as a PyErr.
GrimpError's Debug and Display implementations then call into Python, so
the Rust test binaries started linking libpython, and `just test-rust`
failed wherever libpython isn't on the loader path:
error while loading shared libraries: libpython3.13.so.1.0: cannot
open shared object file: No such file or directory
Make FileSystem::read return Rust errors instead (GrimpError::FileNotFound
and GrimpError::UndecodableFile), and convert them to FileNotFoundError
and UnicodeError where they reach Python, like the other GrimpError
variants. The exception types and messages are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN
As suggested by AI review.
seddonym
force-pushed
the
fix-encoding-declarations
branch
from
October 12, 2026 14:12
70b4a8b to
a87c498
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #322, with a couple of tweaks.