Repository navigation
Fix reader error handling, strengthen tests, rewrite README - #23
Merged
Merged
Conversation
- readBOM counted a read that returned an error as an empty read, so a source that returned io.EOF or a real error after 99 empty reads got io.ErrNoProgress instead. Only (0, nil) now counts as no progress. - detectUtf gave up on UTF-16 whenever probing ended with a non-EOF error, even with all 4 probe bytes read. FE FF 00 41 returned together with an error was Unknown, but the same bytes followed by the error on the next Read were UTF16BigEndian. With 4 bytes read, UTF-32 has already been ruled out, so UTF-16 is now detected in both cases. Both bugs date back to v1.1.1. A 2-3 byte FE FF / FF FE prefix followed by a non-EOF error intentionally still yields Unknown, as before.
- Encoding.String: check exact names, including out-of-range values. - Skip on a *Reader returns the same reader. - Reads with 1-4 byte destination buffers. - FuzzSkip: BOM detection and payload preservation against an independent BOM table, with whole and one-byte source reads. - Benchmarks for small inputs with and without BOM and a 64KB copy.
- README: purpose and boundaries, minimum Go version, one complete example, BOM table, Behavior section, pkg.go.dev badge. - Encoding is a defined type, not a type alias; Read does not search for a BOM on every call; Reader is created by Skip/SkipOnly and does not close the source. - Examples print readable output; new ExampleSkip_utf16 shows that the payload is not transcoded. - The QF1010 lint exclusion is no longer needed.
check runs vet, lint and race tests without touching files. task runs fix and fmt first, then check, so make -j cannot race edits against checks. Targets cover ./... and are all .PHONY.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prepares the next release: two reader-error fixes, stronger tests, a rewritten README and a non-mutating
make check. Exported API,String()names and performance are unchanged.Fixes (both present since v1.1.1)
io.EOFor a real error after 99(0, nil)reads gotio.ErrNoProgressinstead. Only(0, nil)now counts.FE FF 00 41returned together with a non-EOF error wasUnknown(BOM kept), but the same bytes with the error on the nextReadwereUTF16BigEndian. With 4 bytes read, UTF-32 is already ruled out, so UTF-16 is now detected in both cases. The historical 2–3-byteFE FF/FF FE+ non-EOF error case still yieldsUnknown.A differential run of old vs new detection over ~5.7M read schedules found differences only in these two cases.
Tests
Encoding.String()names;Skipon a*Readerreturns the same reader; 1–4 byte destination buffers.FuzzSkipagainst an independent BOM table (EOF-terminated streams, whole and one-byte reads).Docs and tooling
Encodingis a defined type;Readdoes not search for a BOM on each call).ExampleSkip_utf16shows the payload is not transcoded. TheQF1010lint exclusion is no longer needed.make checkonly checks;make taskruns fix, fmt, then check sequentially.Verified locally on Go 1.27.1 and 1.19.13: vet, race + shuffle tests, golangci-lint v2.15.0, 30 s fuzz, README example compiled and run.