diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..8108816 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,143 @@ +# Contributing + +Thanks for your interest in the project. This guide covers the setup, the conventions, and +what a change has to satisfy before it can merge. + +## Development environment + +All dev tooling comes from the Nix flake; `task` wraps every command in `nix develop -c`, so +you never need to enter a shell manually. + +**Prerequisites:** + +- **Nix** with flakes enabled — [Determinate Nix Installer](https://install.determinate.systems/nix) +- **go-task** — `brew install go-task` (global, deliberately not in the flake) +- **Docker or Podman** — for integration tests. Either works; the Taskfile detects which. +- **gh** (optional) — for PRs and CI status + +```bash +git clone https://github.com/jasoet/pkg.git +cd pkg + +task nix:check # verify the toolchain +task docker:check # verify a container runtime +task test +``` + +Run `task --list` to see everything available. **Use `task ` rather than raw `go` +commands** — the tasks carry the flags and environment that CI uses, so "works locally" +means something. + +## Making a change + +1. **Branch off `next`.** v3 development happens there; `main` carries the released state. + Name branches `feat/...`, `fix/...`, `docs/...`, `ci/...`. + +2. **Write the test first.** Every behaviour change needs a test that failed before the fix. + Unit tests carry no build tag, integration tests use `//go:build integration`. + +3. **Verify before you push:** + + ```bash + task check # tests + lint + go vet -tags='example integration argo' ./... # tagged code CI also compiles + ``` + + The tagged vet matters: `task check` only compiles untagged code, and example- or + integration-only breakage has slipped past that gap before. + +4. **Open a PR against `next`** and let CI finish. Merges are squash-only, so the PR title + becomes the commit message — write it as a proper Conventional Commit. + +## Conventions + +### Commit messages + +[Conventional Commits](https://www.conventionalcommits.org/) — they drive releases, so the +type is not cosmetic: + +``` +(): + +[optional body] + +[optional footer] +``` + +Types: `feat`, `fix`, `docs`, `refactor`, `test`, `chore`, `perf`, `ci`, `style`. +Scope is the package (`fix(db): ...`). + +**Breaking changes need a `BREAKING CHANGE:` footer** *and* an entry in +[MIGRATION.md](./MIGRATION.md). Several v3 breaks shipped in `fix:`-typed commits without a +footer and would never have reached the release notes — the migration guide is the backstop, +so update it in the same PR as the break. + +### Package conventions + +Every configurable package must: + +1. Expose a functional-options constructor — `New(opts ...Option) (T, error)` where + construction can fail. +2. Carry `OTelConfig *otel.Config` tagged exactly `yaml:"-" mapstructure:"-"`. +3. Expose `WithOTelConfig(cfg *otel.Config) Option` as the single OTel injection point. +4. Instrument through `otel.Layers.Start*()` at layer boundaries. +5. Back README snippets with `Example*` tests, so documentation cannot drift from a + compiling API. +6. Use testify (`assert`/`require`). + +These are enforced mechanically by `internal/archtest` — **adding a package means extending +its registry**, or the convention test fails. See [CONTEXT.md](./CONTEXT.md) for the +vocabulary and [docs/adr/](./docs/adr/) for why the conventions are shaped this way. + +### Documentation + +Docs are treated as part of the product, not an afterthought. When a change affects the +public API, update in the same PR: + +- the package's `README.md` +- `MIGRATION.md`, if consumers must do something +- `INSTRUCTION.md` / `README.md`, for structural changes +- `AI_PATTERN.md`, which indexes consumer patterns + +Do not write a README snippet you have not compiled. + +### Adding a package + +1. Create `newpkg/`, `newpkg/README.md`, `newpkg/newpkg.go`, `newpkg/newpkg_test.go`. +2. Follow the six package conventions above. +3. Register it in `internal/archtest`. +4. Add an example under `examples/newpkg/` behind `//go:build example`. +5. Update the `README.md` package table and `AI_PATTERN.md`. + +## Testing + +| Tier | Build tag | Command | +|---|---|---| +| Unit | none | `task test` | +| Integration | `integration` | `task test:integration` (needs Docker/Podman) | +| Argo | `argo` | `task test:argo` (needs a k8s cluster with Argo) | +| Examples | `example` | `go build -tags=example ./...` | +| Everything | — | `task test:complete` | + +Integration tests use testcontainers with a 15-minute timeout; always +`defer container.Terminate(ctx)`. + +## Git authorship + +**Commits must be authored solely by a human contributor.** Do not add AI tools as +co-authors, committers, or contributors — no `Co-authored-by`, `Generated by`, or +`Created with` trailers, and no AI attribution in commit messages, PR descriptions, or code +comments. This applies to every commit, including those produced with tool assistance. + +## Versioning + +- **v3** (`next` branch) — the only line receiving features. +- **v2** (`release/v2`) — frozen at v2.13.1, emergency patches only. +- **v1** (`release/v1`) — closed at v1.6.0. + +See [ADR 0001](./docs/adr/0001-freeze-v2-and-ship-v3-as-one-big-bang.md) for the reasoning. + +## Reporting issues + +Include the package, the version, a minimal reproduction, and what you expected instead. +For security issues, please report privately rather than opening a public issue. diff --git a/README.md b/README.md index 15cc687..947e011 100644 --- a/README.md +++ b/README.md @@ -131,7 +131,9 @@ Examples for all packages live in the top-level `examples/` directory (e.g. `./e ## Test Coverage -Coverage combines the unit and integration suites (Argo tests require a k8s cluster and are not included). Per-package figures are shown next to each package in the [Packages](#packages) section above; regenerate the full report with `task test:complete` (writes `output/coverage-all.html`). +Per-package figures are shown next to each package in the [Packages](#packages) section above. They come from the unit **and** integration suites — regenerate with `task test:integration` (needs Docker or Podman; writes `output/coverage-integration.html`). + +Argo tests are excluded from those figures because they need a live k8s cluster with Argo Workflows. To include them, run `task test:complete` (writes `output/coverage-complete.html`). ### Run Tests @@ -144,7 +146,7 @@ task test:integration # Complete test suite with coverage report task test:complete -open output/coverage-all.html +open output/coverage-complete.html ``` ## Key Features @@ -237,7 +239,7 @@ logger := config.GetLogger("my-service") ``` **Features:** Automatic instrumentation, context propagation, graceful shutdown -**Coverage:** 84.8% | **[Examples](./examples/otel/)** | **[Documentation](./otel/README.md)** +**Coverage:** 89.5% | **[Examples](./examples/otel/)** | **[Documentation](./otel/README.md)** #### [config](./config/) - Configuration Management Type-safe YAML configuration with environment variable overrides. @@ -254,7 +256,7 @@ cfg, _ := config.LoadString[AppConfig](yamlContent, "APP") ``` **Features:** Environment variable overrides, nested env vars, generics-based loading -**Coverage:** 97.6% | **[Examples](./examples/config/)** | **[Documentation](./config/README.md)** +**Coverage:** 96.4% | **[Examples](./examples/config/)** | **[Documentation](./config/README.md)** ### Data Access @@ -279,7 +281,7 @@ pool.Find(&users) ``` **Features:** Connection pooling, migrations, OTel tracing, health monitoring -**Coverage:** 76.7% | **[Examples](./examples/db/)** | **[Documentation](./db/README.md)** +**Coverage:** 83.5% | **[Examples](./examples/db/)** | **[Documentation](./db/README.md)** #### [docker](./docker/) - Docker Container Executor Production-ready Docker container management with dual API styles. @@ -313,7 +315,7 @@ exec, _ := docker.NewFromRequest(req) ``` **Features:** Lifecycle management, wait strategies, log streaming, dual API (functional + struct) -**Coverage:** 83.1% | **[Examples](./examples/docker/)** | **[Documentation](./docker/README.md)** +**Coverage:** 84.1% | **[Examples](./examples/docker/)** | **[Documentation](./docker/README.md)** #### [argo](./argo/) - Argo Workflows Client Production-ready Argo Workflows client with flexible configuration. @@ -343,7 +345,7 @@ ctx, client, err := argo.NewClientWithOptions(ctx, ``` **Features:** Multiple connection modes, functional options, OTel support, proper error handling -**[Examples](./examples/argo/)** | **[Documentation](./argo/README.md)** +**Coverage:** 94.8% | **[Examples](./examples/argo/)** | **[Documentation](./argo/README.md)** #### [retry](./retry/) - Retry with Exponential Backoff Production-ready retry mechanism using `cenkalti/backoff/v4` with OTel instrumentation. @@ -364,7 +366,7 @@ return retry.Permanent(fmt.Errorf("invalid config")) ``` **Features:** Exponential backoff, context-aware, OTel tracing, permanent error marking -**[Documentation](./retry/README.md)** +**Coverage:** 100.0% | **[Examples](./examples/retry/)** | **[Documentation](./retry/README.md)** #### [base32](./base32/) - Crockford Base32 Encoding Crockford Base32 encoding with CRC-10 checksums for human-readable, error-correcting identifiers. @@ -386,7 +388,7 @@ normalized := base32.NormalizeBase32("ab-CD iL o9") // "ABCD1109" ``` **Features:** URL-safe alphabet, automatic error correction, CRC-10 checksums, compact encoding -**[Examples](./examples/base32/)** | **[Documentation](./base32/README.md)** +**Coverage:** 100.0% | **[Examples](./examples/base32/)** | **[Documentation](./base32/README.md)** ### HTTP & gRPC @@ -416,7 +418,7 @@ if err := srv.Start(); err != nil { // blocks until srv.Shutdown(ctx) ``` **Features:** Health checks, graceful shutdown, middleware -**Coverage:** 77.1% | **[Examples](./examples/server/)** | **[Documentation](./server/README.md)** +**Coverage:** 97.0% | **[Examples](./examples/server/)** | **[Documentation](./server/README.md)** #### [grpc](./grpc/) - gRPC Server Production-ready gRPC with Echo gateway integration. @@ -435,7 +437,7 @@ server.Start() ``` **Features:** H2C mode, dual HTTP/gRPC, gateway, observability -**Coverage:** 71.2% | **[Examples](./examples/grpc/)** | **[Documentation](./grpc/README.md)** +**Coverage:** 82.0% | **[Examples](./examples/grpc/)** | **[Documentation](./grpc/README.md)** #### [rest](./rest/) - HTTP Client Resilient REST client with OTel tracing. @@ -457,7 +459,7 @@ response, _ := client.MakeRequestWithTrace(ctx, "GET", url, "", headers) ``` **Features:** Retries, tracing, middleware support -**Coverage:** 92.9% | **[Examples](./examples/rest/)** | **[Documentation](./rest/README.md)** +**Coverage:** 93.0% | **[Examples](./examples/rest/)** | **[Documentation](./rest/README.md)** ### Utilities @@ -478,7 +480,7 @@ results, _ := concurrent.ExecuteConcurrently(ctx, funcs) ``` **Features:** Go 1.26+ generics, error aggregation, context support -**Coverage:** 95.1% | **[Examples](./examples/concurrent/)** | **[Documentation](./concurrent/README.md)** +**Coverage:** 100.0% | **[Examples](./examples/concurrent/)** | **[Documentation](./concurrent/README.md)** #### [temporal](./temporal/) - Workflow Orchestration Temporal workflow integration with observability. @@ -501,7 +503,7 @@ handle, _ := manager.CreateWorkflowSchedule(ctx, "daily-job", temporal.WorkflowS ``` **Features:** Schedule management, workers, job definitions, monitoring -**Coverage:** 81.2% | **[Examples](./examples/temporal/)** | **[Documentation](./temporal/README.md)** +**Coverage:** 84.5% | **[Examples](./examples/temporal/)** | **[Documentation](./temporal/README.md)** #### [ssh](./ssh/) - SSH Tunneling Secure SSH tunneling and port forwarding. @@ -523,7 +525,7 @@ defer tunnel.Close() ``` **Features:** Port forwarding, connection pooling, error handling -**Coverage:** 78.2% | **[Examples](./examples/ssh/)** | **[Documentation](./ssh/README.md)** +**Coverage:** 85.6% | **[Examples](./examples/ssh/)** | **[Documentation](./ssh/README.md)** #### [compress](./compress/) - File Compression Secure file compression with validation. @@ -540,7 +542,7 @@ compress.TarGz("/path/to/directory", outputFile) ``` **Features:** gzip, tar.gz, security validation, path traversal protection -**Coverage:** 82.4% | **[Examples](./examples/compress/)** | **[Documentation](./compress/README.md)** +**Coverage:** 85.3% | **[Examples](./examples/compress/)** | **[Documentation](./compress/README.md)** ## Contributing diff --git a/docker/README.md b/docker/README.md index 7cc231e..030df2f 100644 --- a/docker/README.md +++ b/docker/README.md @@ -1026,6 +1026,5 @@ if err := exec.Start(ctx); err != nil { ## Related Packages -- **[otel](../otel/)** - OpenTelemetry v2 configuration and utilities +- **[otel](../otel/)** - OpenTelemetry configuration, utilities, and structured logging - **[config](../config/)** - Configuration management with validation -- **[logging](../logging/)** - Structured logging with context diff --git a/docs/plans/2026-07-22-v3-audit-backlog.md b/docs/plans/2026-07-22-v3-audit-backlog.md index 712d40d..0dee7f2 100644 --- a/docs/plans/2026-07-22-v3-audit-backlog.md +++ b/docs/plans/2026-07-22-v3-audit-backlog.md @@ -127,29 +127,30 @@ Enforced mechanically by `internal/archtest` (Phase 1). - Add golden checksum regression tests. - `AppendChecksum`/`ValidateChecksum` should normalize input or loudly document caller must. +## Resolved Process Items + +Kept for provenance; each records where the fix landed. + +- ~~**gorelease blocks the next→main v3 merge.**~~ RESOLVED — `github.head_ref != 'next'` added to the blocking condition and `head_ref == 'next'` to the informational one (PR #59). Baseline story: gorelease reports "Inferred base version: none / Suggested version: v3.0.0" until the first non-prerelease `/v3` tag exists on main, which is why the release PR must not gate on it. +- ~~**gorelease is unpinned (`@latest`).**~~ RESOLVED — pinned to the `golang.org/x/exp` pseudo-version already in go.mod, via a workflow-level `GORELEASE_VERSION` env var (PR #59). +- ~~**`.releaserc.json` headerPartial hardcodes `/v2`.**~~ RESOLVED — now emits `go get github.com/jasoet/pkg/v3@{{currentTag}}`. +- ~~**Per-commit gate misses build-tagged files.**~~ RESOLVED — ci.yml runs `go vet -tags='example integration argo' ./...` as its own step. +- ~~**Docs phase named checkbox:** sweep examples/db and examples/rest for deleted-logging references.~~ RESOLVED — verified: the remaining "logging" hits in both files refer to the activity, not the deleted package. The real stale reference was `docker/README.md`'s Related Packages link to `../logging/`, now removed. +- ~~**Conventions doc:** constructor naming split.~~ RESOLVED as **not a defect** — see [ADR 0003](../adr/0003-constructor-naming.md). `New` returns the package's primary type; `New` names it when a package builds several. `retry.New` returns a `Config` because a `Config` *is* retry's primary artifact. +- ~~**Migration guide** items (rest, temporal, grpc, server, ssh, argo, db, retry, config sections).~~ RESOLVED — all folded into `MIGRATION.md`, including the breaks that shipped without `BREAKING CHANGE` footers. Two backlog notes described a planned API that never shipped and are documented as built: `temporal.NewClient` takes no ctx, and argo operations dropped the namespace parameter as well as the `*otel.Config` one. +- ~~**Final docs sweep must cover stale db APIs**~~ RESOLVED — verified no `cfg.Pool()` or `RunPostgresMigrationsWithGorm` references remain outside the historical plan documents. +- ~~**Root README coverage figures are stale.**~~ RESOLVED — regenerated from a real unit+integration run; `argo`, `retry` and `base32` gained the figures they never had. +- ~~**docker NetworkSettings parity:** unguarded derefs in network.go.~~ RESOLVED — guarded via pure projection helpers with unit tests for the nil case (PR #61). +- ~~**Shared gap (server + grpc):** neither HTTP tracing middleware extracts W3C traceparent.~~ RESOLVED — both now extract, with tests pinning both the joined-trace and root-span cases (PR #60). +- ~~**argo v3.x (1) hard-coded 5s poll interval.**~~ RESOLVED — `argo.WithPollInterval(d)`. +- ~~**argo v3.x (2) non-sentinel errors.**~~ RESOLVED — `ErrWaitTimeout` and `ErrWorkflowFailed`. + ## Open Process Items (from Phase 1 review) -- **gorelease blocks the next→main v3 merge.** The blocking API check (ci.yml) fires on a `next`→`main` PR (base_ref=main) and will report the intended v3 breaks as incompatibilities; after the module path becomes `/v3`, gorelease has no prior v3 baseline. The final phase must deliberately handle this: add `&& github.head_ref != 'next'` to the blocking condition as part of the v3 merge PR, and decide the gorelease baseline story for `/v3`. -- **gorelease is unpinned (`@latest`).** A blocking gate floating on latest is non-reproducible. Pin a version or add gorelease to flake.nix and run the flake-provided binary. -- **`.releaserc.json` headerPartial hardcodes `/v2`** in the `go get` line — must become `/v3` when the module path bumps. -- **Per-commit gate misses build-tagged files.** task check compiles only untagged code; a tagged-only break (example/integration) slipped through Phase 3 Task 1. Remaining phases: include `go build -tags=example,integration ./...` in verification. -- **Docs phase named checkbox:** sweep `examples/db/README.md` and `examples/rest/README.md` for deleted-logging references (lines ~38, ~392, ~499-504). -- **Migration guide must note:** retry RandomizationFactor range widened from [0,1) to [0,1] (1.0 now valid); config keyDepth is prefix-relative (see config/README.md migration note). - **Post-v3 consideration:** seal config.Option (interface with unexported apply) to fully hide viper from godoc, or explicitly accept the leak; add archtest ratchet for third-party types in public signatures. -- **Conventions doc:** constructor naming split — otel.NewConfig/server.NewConfig vs retry.New/grpc.New/docker.New. Pick one in the v3 conventions writeup. -- **Migration guide (rest section) must disclose:** Client.HandleResponse was unexported in Phase 5 (commit 6cc5af1) without a BREAKING CHANGE footer mention. Guide text: typed errors for non-2xx now come from MakeRequest/MakeRequestWithTrace directly (the returned *rest.Response is non-nil on HTTP errors, so status/body remain inspectable); GetRestClient escape-hatch users who relied on HandleResponse must write their own status mapping. -- **Migration guide (temporal section):** config-passing constructors → temporal.NewClient(temporal.WithConfig(cfg)) or options (WithHostPort/WithNamespace/WithOTelConfig). Managers now borrow a caller-owned client.Client — consumers who used NewWorkerManager(&cfg)/NewScheduleManager(cfg)+Close must now create AND close their own client (managers no longer close it). WorkflowManager.Close REMOVED entirely (was a no-op after ownsClient removal). Close() → Close(ctx) on WorkerManager and ScheduleManager. NewWorkflowManagerWithNamespace's namespace param is now always authoritative (previously ignored when a *Config was passed). NOTE: commit cadc208's BREAKING footer omits the WorkflowManager.Close removal — release notes must include it manually. -- **Final docs sweep must cover stale db APIs** in: PROJECT_TEMPLATE.md (lines ~328,361,580,599,1196,1200,2029,2030,2100,2106), AI_PATTERN.md (~106), examples/fullstack-otel/README.md (~213). Old: cfg.Pool() and RunPostgresMigrationsWithGorm. New: db.NewPool(db.WithConnectionConfig(cfg)); gormDB.DB() + RunPostgresMigrations. -- **Migration guide (db section):** metrics-only configs now emit db.client.connections.* series that previously never appeared (bug fix working, dashboards may newly fire); RedactedDsn output changes only for pathological password/DSN-substring collisions (safe direction). -- **docker remaining leaks (decision needed):** `Executor.Inspect() (*container.InspectResponse, error)` (status.go:166) and `Executor.GetStats() (container.StatsResponseReader, error)` (status.go:258) still expose docker/docker types. Decide: document as escape hatch by design (like temporal/argo) or wrap in v3.x. +- ~~**docker remaining leaks (decision needed).**~~ DECIDED — `Executor.Inspect()` and `GetStats()` stay as documented escape hatches; see [ADR 0004](../adr/0004-selective-de-leak-of-third-party-types.md). Wrapping them would mean tracking the docker API for information that has no natural library-owned shape. - **docker ContainerTarget limits:** Logs() hardcodes Follow/Timestamps off; exec-based readiness strategies (pg_isready via ContainerExec) are no longer expressible via WaitForFunc — migration guide must note such consumers construct their own client. Consider Exec-capable target in v3.x. -- **docker NetworkSettings parity:** unguarded derefs in network.go Executor methods (MappedPort, GetAllPorts, GetNetworks, GetIPAddress) — pre-existing; guard pattern established in target.go State(). -- **Migration guide (grpc section) must disclose** (shipped in fix-type commits without footers): (a) MountGatewayOnEcho now strips the base path — external callers who registered mux patterns including the prefix must switch to proto-relative patterns; (b) Start/StartH2C/StartSeparate now return nil instead of http.ErrServerClosed on clean shutdown — drop errors.Is(err, http.ErrServerClosed) special-casing; (c) GetGRPCServer() returns nil after Stop until the next Start. -- **Migration guide (server section):** Start/StartWithConfig/DefaultConfig → server.New(opts...) + srv.Start()/srv.Shutdown(ctx). SIGNAL HANDLING REMOVED — old API blocked on SIGINT/SIGTERM and drained gracefully; consumers must wire their own signal.Notify → Shutdown or lose graceful termination (biggest consumer-facing change of the phase). OTelConfig now auto-emits http.server.* spans/metrics (new telemetry series appear for existing users). Restart-after-shutdown errors; Shutdown is idempotent; port validation now fails at New. -- **Conventions writeup:** grpc supports Start/Stop/Start cycles (rebuild), server forbids restart — deliberate divergence to document. Server and grpc gateway emit same-named http.server.* metrics with different attribute sets (server: method+status; gateway: +http.route +active_requests) — align or document. -- **Shared gap (server + grpc):** neither HTTP tracing middleware extracts W3C traceparent from inbound headers — inbound spans are always roots. Cover both packages together in v3.x. -- **Root README coverage figures are stale across packages** (e.g. server shows 77.1%, now 97.5%) — refresh in final docs phase. -- **ssh fix-wave items (Phase 10):** (1) forward() lacks half-close — Close() blocks ~90s on keep-alive clients (godoc-documented at tunnel.go:267-269; integration tests work around with DisableKeepAlives). Fix in the phase fix wave. (2) forward() still uses hardcoded nil-config LogHelper (tunnel.go:287) — last nil-otel site; needs per-connection ctx propagation from the Start span. (3) Close no-op path leaves span status Unset. (4) Close error text now wraps client-close error (migration note). -- **ssh v3.x items (from Phase 10 final review):** (1) accept-loop races: accept goroutine reads t.stopCh unlocked while Start reassigns it (race + possible 100% CPU spin after Start→Close→Start); t.wg.Add after Accept races with Close's wg.Wait — capture stopCh locally, exit on persistent Accept error, guard Add. (2) Residual Close hang for peers ignoring FIN — consider tracking active local conns and force-closing them in Close. (3) Pin the ssh testcontainer image digest (currently :latest + unconditional 5s sleep). (4) forward() shipped config-aware logging (context.Background) instead of Start-span ctx propagation — descoped decision, logs carry no trace correlation. -- **Migration guide (ssh section):** New is variadic (source-compatible); Close error text now wrapped "SSH client close error: %w"; Close now tears down in-flight forwarded connections immediately instead of draining (finish work before calling Close); half-close propagation gives peers prompt EOF (observable for streaming protocols); secrets (Password/PrivateKey/PrivateKeyPassphrase) are yaml:"-" by design — inject via env/code, YAML password keys are silently dropped. -- **argo v3.x items (from Phase 12 final review):** (1) hard-coded 5s poll interval in SubmitAndWait (operations.go:125) — make configurable. (2) Non-sentinel errors (timeout at operations.go:136, workflow-failed at :171) — make wrappable with errors.Is. (3) Migration guide (argo section): Option signature change; operations dropped positional *otel.Config — thread the NewClient-returned ctx (fresh context.Background() silently disables instrumentation); in-cluster Namespace trimming fixed (in-cluster mode may have been silently broken before). +- ~~**Conventions writeup:** grpc restart cycles vs server single-use; `http.server.*` attribute-set divergence.~~ DOCUMENTED — [ADR 0005](../adr/0005-lifecycle-divergence-between-server-and-grpc.md) and the telemetry table in `MIGRATION.md`. Both divergences are deliberate and stay. +- **ssh fix-wave items (Phase 10)** — (1) half-close: RESOLVED, `forward()` propagates `CloseWrite`. (2) nil-config LogHelper: PARTIALLY RESOLVED — `forward()`/`acceptLoop()` are now config-aware, but still build their context from `context.Background()`, so logs carry no trace correlation (see v3.x item 4 below). (3) Close no-op path span status: still open. (4) Close error wrapping: RESOLVED and noted in `MIGRATION.md`. +- **ssh v3.x items (from Phase 10 final review):** (1) accept-loop races — PARTIALLY RESOLVED. The busy-spin half is fixed (capped exponential backoff, exit on `net.ErrClosed`). **Still open:** `acceptLoop` reads `t.stopCh` unlocked at three points while `Start` reassigns it under `t.mu` (tunnel.go:302,328,340 vs :168), and `t.wg.Add` after Accept still races `Close`'s `wg.Wait`. Verified still present as of this sweep — capture `stopCh` locally at loop entry and guard the `Add`. (2) Residual Close hang for peers ignoring FIN — consider tracking active local conns and force-closing them in Close. (3) Pin the ssh testcontainer image digest (currently :latest + unconditional 5s sleep). (4) forward() shipped config-aware logging (context.Background) instead of Start-span ctx propagation — descoped decision, logs carry no trace correlation. +- **argo:** all Phase 12 follow-ups resolved (configurable poll interval, sentinel errors, migration notes). Nothing open. diff --git a/examples/compress/README.md b/examples/compress/README.md index d2f1d23..bbf0c41 100644 --- a/examples/compress/README.md +++ b/examples/compress/README.md @@ -4,7 +4,7 @@ This directory contains examples demonstrating how to use the `compress` package ## 📍 Example Code Location -**Full example implementation:** [/compress/examples/example.go](https://github.com/jasoet/pkg/blob/main/compress/examples/example.go) +**Full example implementation:** [`example.go`](./example.go) ## 🚀 Quick Reference for LLMs/Coding Agents @@ -50,10 +50,11 @@ The `compress` package provides utilities for: ## Running the Examples -To run the examples, use the following command from the `compress/examples` directory: +Run from the repository root. The `example` build tag is required — without it the +file is excluded from the build: ```bash -go run example.go +go run -tags=example ./examples/compress ``` This will: @@ -63,7 +64,7 @@ This will: ## Example Descriptions -The [example.go](https://github.com/jasoet/pkg/blob/main/compress/examples/example.go) file demonstrates several use cases: +The [example.go](./example.go) file demonstrates several use cases: ### 1. Creating a tar archive diff --git a/examples/grpc/README.md b/examples/grpc/README.md index 8e2d2e1..d065427 100644 --- a/examples/grpc/README.md +++ b/examples/grpc/README.md @@ -348,4 +348,4 @@ GRPC_GO_LOG_VERBOSITY_LEVEL=2 GRPC_GO_LOG_SEVERITY_LEVEL=info \ - [gRPC Go Documentation](https://grpc.io/docs/languages/go/) - [Protocol Buffers Guide](https://developers.google.com/protocol-buffers) - [gRPC Streaming Guide](https://grpc.io/docs/what-is-grpc/core-concepts/#server-streaming-rpc) -- [pkg/grpc Package](../README.md) - Our reusable gRPC component \ No newline at end of file +- [pkg/grpc Package](../../grpc/README.md) - Our reusable gRPC component \ No newline at end of file diff --git a/examples/otel/README.md b/examples/otel/README.md index 459d9c4..3e6a72e 100644 --- a/examples/otel/README.md +++ b/examples/otel/README.md @@ -1,389 +1,116 @@ # OTel Package Examples -This directory contains examples demonstrating how to use the `otel` package for OpenTelemetry instrumentation configuration in Go applications. +Runnable examples for the [`otel`](../../otel/) package — the configuration hub every other +package takes its telemetry from. -## 📍 Example Code Location +**Example code:** [`example.go`](./example.go) -**Full example implementation:** [/otel/examples/example.go](https://github.com/jasoet/pkg/blob/main/otel/examples/example.go) +## Running -## 🚀 Quick Reference for LLMs/Coding Agents +```bash +go run -tags=example ./examples/otel +``` + +The `example` build tag is required; without it the file is excluded from the build. + +## Quick reference ```go -// Basic usage pattern -import "github.com/jasoet/pkg/v2/otel" +import "github.com/jasoet/pkg/v3/otel" -// Create configuration with all pillars -cfg := &otel.Config{ - TracerProvider: tracerProvider, // For distributed tracing - MeterProvider: meterProvider, // For metrics - LoggerProvider: loggerProvider, // For structured logging - ServiceName: "my-service", - ServiceVersion: "1.0.0", -} +// Configuration is built with functional options. +cfg := otel.NewConfig("my-service", + otel.WithServiceVersion("1.0.0"), + otel.WithTracerProvider(tracerProvider), + otel.WithMeterProvider(meterProvider), + otel.WithLoggerProvider(loggerProvider), +) -// Check what's enabled +// Ask before you use — but the getters are safe either way. if cfg.IsTracingEnabled() { tracer := cfg.GetTracer("scope-name") } -// Use LogHelper for OTel-aware logging +// LogHelper correlates logs with the active span. logger := otel.NewLogHelper(ctx, cfg, "github.com/myorg/myapp", "myFunc") logger.Info("Processing request", otel.F("request_id", reqID)) logger.Error(err, "Failed to process", otel.F("user_id", userID)) - -// No-op mode (zero overhead when nil) -cfg := &otel.Config{ServiceName: "my-service"} // All providers nil = no-op -``` - -**Key Features:** -- Centralized configuration for traces, metrics, and logs -- No-op implementations when providers are nil (zero overhead) -- LogHelper for automatic log-trace correlation -- Selective enabling of telemetry pillars - -## Overview - -The `otel` package provides: -- **Centralized Configuration**: Single config struct for all OTel providers -- **Telemetry Pillars**: Independent control of traces, metrics, and logs -- **LogHelper**: OTel-aware logging with automatic trace correlation -- **No-op Support**: Zero overhead when telemetry is disabled - -## Running the Examples - -To run the examples, use the following command from the repository root: - -```bash -go run ./examples/otel -``` - -Or from the `otel/examples` directory: - -```bash -go run example.go -``` - -This will demonstrate: -1. Basic OTel configuration setup -2. No-op configuration (telemetry disabled) -3. LogHelper usage patterns -4. Selective telemetry pillar enabling -5. Configuration validation - -## Example Descriptions - -The [example.go](https://github.com/jasoet/pkg/blob/main/otel/examples/example.go) file demonstrates several configuration patterns: - -### 1. Basic Configuration - -Shows how to create a complete OTel configuration: - -```go -cfg := &otel.Config{ - TracerProvider: tracerProvider, - MeterProvider: meterProvider, - LoggerProvider: loggerProvider, - ServiceName: "example-service", - ServiceVersion: "1.0.0", -} - -// Check what's enabled -fmt.Println(cfg.IsTracingEnabled()) // true -fmt.Println(cfg.IsMetricsEnabled()) // true -fmt.Println(cfg.IsLoggingEnabled()) // true -``` - -### 2. No-op Configuration - -Demonstrates zero-overhead telemetry when disabled: - -```go -// All providers nil = no-op mode -cfg := &otel.Config{ - ServiceName: "example-service", - ServiceVersion: "1.0.0", -} - -fmt.Println(cfg.IsTracingEnabled()) // false -fmt.Println(cfg.IsMetricsEnabled()) // false -fmt.Println(cfg.IsLoggingEnabled()) // false - -// No overhead - safe to use in production -``` - -### 3. LogHelper Usage - -Shows OTel-aware logging patterns: - -```go -// Without OTel (falls back to zerolog) -logger1 := otel.NewLogHelper(ctx, nil, "", "example.doWork") -logger1.Info("Starting work", otel.F("worker_id", 123)) - -// With OTel (automatic trace_id/span_id injection) -logger2 := otel.NewLogHelper(ctx, cfg, "github.com/myorg/myapp", "example.processData") -logger2.Info("Data processed", - otel.F("records_processed", 1000), - otel.F("duration_ms", 250)) - -// Error logging -logger2.Error(err, "Processing failed", - otel.F("error_code", "E001"), - otel.F("retry_count", 3)) -``` - -### 4. Telemetry Pillars - -Demonstrates selective enabling of telemetry: - -```go -// Scenario 1: Traces only -cfg1 := &otel.Config{ - TracerProvider: tracerProvider, - ServiceName: "tracing-only-service", -} - -// Scenario 2: Metrics only -cfg2 := &otel.Config{ - MeterProvider: meterProvider, - ServiceName: "metrics-only-service", -} - -// Scenario 3: All pillars -cfg3 := &otel.Config{ - TracerProvider: tracerProvider, - MeterProvider: meterProvider, - LoggerProvider: loggerProvider, - ServiceName: "full-telemetry-service", -} ``` -### 5. Configuration Validation +> **`NewConfig` installs a default zerolog-backed logger provider** unless you opt out with +> `otel.WithoutLogging()`. A config with no options is therefore *not* fully no-op — logging +> is on. Use `WithoutTracing()` / `WithoutMetrics()` / `WithoutLogging()` to turn pillars off +> explicitly. -Shows safe configuration checking: +## What the example covers -```go -// Check before use -if cfg.IsTracingEnabled() { - tracer := cfg.GetTracer("my-scope") - // Use tracer... -} +| # | Example | Shows | +|---|---|---| +| 1 | Basic configuration | All three pillars wired from SDK providers | +| 2 | No-op configuration | `WithoutLogging()` to get a genuinely inert config | +| 3 | LogHelper usage | Zerolog fallback vs. OTel-backed, and `otel.F()` fields | +| 4 | Telemetry pillars | Traces-only, metrics-only, and all-on configurations | +| 5 | Configuration validation | `Is*Enabled()` checks and safe no-op getters | -// Safe to call even if not enabled (returns no-op) -meter := cfg.GetMeter("my-scope") -``` +## Telemetry pillars -## Telemetry Pillars - -The otel package supports three independent telemetry pillars: - -### 1. Traces (Distributed Tracing) +The three pillars are independent — enable only what you need. ```go -cfg := &otel.Config{ - TracerProvider: tracerProvider, - ServiceName: "my-service", -} +// Traces only +cfg := otel.NewConfig("my-service", + otel.WithTracerProvider(tracerProvider), + otel.WithoutLogging(), +) -tracer := cfg.GetTracer("my-scope") -// Use for distributed tracing... +// Metrics only +cfg := otel.NewConfig("my-service", + otel.WithMeterProvider(meterProvider), + otel.WithoutLogging(), +) ``` -### 2. Metrics (Measurements) +When a provider is absent, `Is*Enabled()` returns `false` and `GetTracer`/`GetMeter`/ +`GetLogger` return no-op implementations. Calling them is always safe — the `Is*Enabled()` +check is for skipping *your* work, not for guarding the getters. -```go -cfg := &otel.Config{ - MeterProvider: meterProvider, - ServiceName: "my-service", -} - -meter := cfg.GetMeter("my-scope") -// Use for metrics collection... -``` +## Passing the config to other packages -### 3. Logs (Structured Logging) +One config, injected everywhere via each package's `WithOTelConfig` option: ```go -cfg := &otel.Config{ - LoggerProvider: loggerProvider, - ServiceName: "my-service", -} +otelCfg := otel.NewConfig("my-service", + otel.WithServiceVersion(version), + otel.WithTracerProvider(tp), + otel.WithMeterProvider(mp), +) -logger := cfg.GetLogger("my-scope") -// Use for structured logging with trace correlation... +srv, err := server.New(server.WithPort(8080), server.WithOTelConfig(otelCfg)) +grpcSrv, err := grpc.New(grpc.WithGRPCPort("50051"), grpc.WithOTelConfig(otelCfg)) +client := rest.NewClient(rest.WithOTelConfig(otelCfg)) +pool, err := db.NewPool(db.WithConnectionConfig(dbCfg), db.WithOTelConfig(otelCfg)) ``` -## LogHelper Features - -The LogHelper provides: -- **Automatic Trace Correlation**: Logs include trace_id and span_id -- **Fallback to Zerolog**: Works without OTel configuration -- **Structured Logging**: Type-safe field addition with `F()` function -- **Multiple Log Levels**: Debug, Info, Warn, Error +`OTelConfig` is never read from YAML — it is tagged `yaml:"-" mapstructure:"-"` on every +config struct, so it must be injected in code. See +[ADR 0002](../../docs/adr/0002-otel-config-is-injected-never-serialized.md). -### Basic Usage +## LogHelper ```go logger := otel.NewLogHelper(ctx, cfg, "github.com/myorg/myapp", "functionName") -// Info logging -logger.Info("User logged in", - otel.F("user_id", 123), - otel.F("email", "user@example.com")) - -// Debug logging -logger.Debug("Cache lookup", - otel.F("cache_key", "user:123"), - otel.F("hit", true)) - -// Warning logging -logger.Warn("Rate limit approaching", - otel.F("current_rate", 95), - otel.F("max_rate", 100)) - -// Error logging -logger.Error(err, "Failed to process payment", - otel.F("payment_id", "PAY-123"), - otel.F("amount", 99.99)) -``` - -## Integration Patterns - -### With Server Package - -```go -serverConfig := &server.Config{ - OtelConfig: &otel.Config{ - TracerProvider: tracerProvider, - ServiceName: "my-service", - ServiceVersion: "1.0.0", - }, -} -``` - -### With GRPC Package - -```go -grpcOptions := []grpc.ServerOption{ - grpc.OtelConfig(&otel.Config{ - TracerProvider: tracerProvider, - ServiceName: "my-grpc-service", - }), -} -``` - -## Best Practices - -### 1. Centralize OTel Configuration - -```go -// Create once, reuse everywhere -otelCfg := &otel.Config{ - TracerProvider: tracerProvider, - MeterProvider: meterProvider, - LoggerProvider: loggerProvider, - ServiceName: "my-service", - ServiceVersion: version, -} - -// Pass to all components -serverCfg.OtelConfig = otelCfg -grpcCfg.OtelConfig = otelCfg -``` - -### 2. Always Check Before Using - -```go -if cfg.IsTracingEnabled() { - tracer := cfg.GetTracer("my-scope") - // Safe to use tracer -} -``` - -### 3. Use LogHelper Consistently - -```go -// Create logger for each function/scope -logger := otel.NewLogHelper(ctx, cfg, scopeName, functionName) - -// Use throughout function -logger.Debug("Starting...") -logger.Info("Processing...") -logger.Error(err, "Failed...") -``` - -### 4. Structured Logging with F() - -```go -// Type-safe field addition -logger.Info("Event occurred", - otel.F("event_id", 123), - otel.F("user_id", userID), - otel.F("timestamp", time.Now())) -``` - -### 5. Enable Selectively - -```go -// Development: All pillars -devCfg := &otel.Config{ - TracerProvider: tp, - MeterProvider: mp, - LoggerProvider: lp, -} - -// Production: Traces and metrics only -prodCfg := &otel.Config{ - TracerProvider: tp, - MeterProvider: mp, -} +logger.Debug("Cache lookup", otel.F("cache_key", "user:123"), otel.F("hit", true)) +logger.Info("User logged in", otel.F("user_id", 123)) +logger.Warn("Rate limit approaching", otel.F("current_rate", 95)) +logger.Error(err, "Failed to process payment", otel.F("payment_id", "PAY-123")) ``` -## Configuration Options - -### Required Fields - -- `ServiceName` - Name of your service - -### Optional Fields - -- `ServiceVersion` - Version of your service -- `TracerProvider` - For distributed tracing (nil = no-op) -- `MeterProvider` - For metrics (nil = no-op) -- `LoggerProvider` - For logging (nil = no-op) - -## No-op Behavior - -When providers are nil: -- `IsTracingEnabled()` returns `false` -- `IsMetricsEnabled()` returns `false` -- `IsLoggingEnabled()` returns `false` -- `GetTracer()` returns no-op tracer -- `GetMeter()` returns no-op meter -- `GetLogger()` returns no-op logger -- **Zero overhead** - no performance impact - -This allows easy toggling of telemetry without code changes. - -## Differences from fullstack-otel - -The `examples/fullstack-otel` directory contains a complete application with: -- Full OpenTelemetry setup -- Jaeger/Prometheus integration -- Multi-service architecture -- Production-ready configuration - -The `examples/otel` directory focuses on: -- Basic configuration patterns -- Simple usage examples -- LogHelper demonstration -- Configuration validation - -For production setup guidance, see `examples/fullstack-otel`. +Passing `nil` for the config falls back to zerolog, so helper code works with or without +telemetry configured. With a config, logs carry `trace_id` and `span_id` for correlation. -## Further Reading +## Related -- [OTel Package Documentation](https://github.com/jasoet/pkg/tree/main/otel) -- [Fullstack OTel Example](https://github.com/jasoet/pkg/tree/main/examples/fullstack-otel) -- [OpenTelemetry Go Documentation](https://opentelemetry.io/docs/instrumentation/go/) -- [Main pkg Repository](https://github.com/jasoet/pkg) +- [`otel` package documentation](../../otel/README.md) +- [fullstack-otel example](../fullstack-otel/) — complete app with Jaeger and Prometheus +- [OpenTelemetry Go docs](https://opentelemetry.io/docs/languages/go/) diff --git a/examples/otel/example.go b/examples/otel/example.go index 7a5baa5..e8d0986 100644 --- a/examples/otel/example.go +++ b/examples/otel/example.go @@ -8,7 +8,7 @@ // - No-op vs active provider configurations // - Integration patterns // -// Run with: go run ./examples/otel +// Run with: go run -tags=example ./examples/otel package main import ( @@ -50,13 +50,12 @@ func basicConfiguration() { loggerProvider := log.NewLoggerProvider() // Create OTel configuration - cfg := &otel.Config{ - TracerProvider: tracerProvider, - MeterProvider: meterProvider, - LoggerProvider: loggerProvider, - ServiceName: "example-service", - ServiceVersion: "1.0.0", - } + cfg := otel.NewConfig("example-service", + otel.WithServiceVersion("1.0.0"), + otel.WithTracerProvider(tracerProvider), + otel.WithMeterProvider(meterProvider), + otel.WithLoggerProvider(loggerProvider), + ) fmt.Printf("Service Name: %s\n", cfg.ServiceName) fmt.Printf("Service Version: %s\n", cfg.ServiceVersion) @@ -74,11 +73,13 @@ func basicConfiguration() { func noopConfiguration() { fmt.Println("--- Example 2: No-op Configuration (Telemetry Disabled) ---") - // Configuration with nil providers - no overhead - cfg := &otel.Config{ - ServiceName: "example-service", - ServiceVersion: "1.0.0", - } + // No providers configured - no overhead. + // NewConfig installs a default zerolog-backed logger provider unless you + // opt out, so WithoutLogging is what makes all three pillars no-ops. + cfg := otel.NewConfig("example-service", + otel.WithServiceVersion("1.0.0"), + otel.WithoutLogging(), + ) fmt.Printf("Service Name: %s\n", cfg.ServiceName) fmt.Printf("Service Version: %s\n", cfg.ServiceVersion) @@ -106,11 +107,10 @@ func logHelperUsage() { // Scenario 2: With OTel configured fmt.Println("\n2. With OTel configured:") loggerProvider := log.NewLoggerProvider() - cfg := &otel.Config{ - LoggerProvider: loggerProvider, - ServiceName: "example-service", - ServiceVersion: "1.0.0", - } + cfg := otel.NewConfig("example-service", + otel.WithServiceVersion("1.0.0"), + otel.WithLoggerProvider(loggerProvider), + ) logger2 := otel.NewLogHelper(ctx, cfg, "github.com/jasoet/pkg/v3/example", "example.processData") logger2.Info("Data processed successfully", @@ -139,10 +139,10 @@ func telemetryPillars() { // Scenario 1: Traces only fmt.Println("1. Traces Only:") tracerProvider := trace.NewTracerProvider() - cfg1 := &otel.Config{ - TracerProvider: tracerProvider, - ServiceName: "tracing-service", - } + cfg1 := otel.NewConfig("tracing-service", + otel.WithTracerProvider(tracerProvider), + otel.WithoutLogging(), + ) fmt.Printf(" Tracing: %v, Metrics: %v, Logging: %v\n", cfg1.IsTracingEnabled(), cfg1.IsMetricsEnabled(), @@ -152,10 +152,10 @@ func telemetryPillars() { // Scenario 2: Metrics only fmt.Println("\n2. Metrics Only:") meterProvider := metric.NewMeterProvider() - cfg2 := &otel.Config{ - MeterProvider: meterProvider, - ServiceName: "metrics-service", - } + cfg2 := otel.NewConfig("metrics-service", + otel.WithMeterProvider(meterProvider), + otel.WithoutLogging(), + ) fmt.Printf(" Tracing: %v, Metrics: %v, Logging: %v\n", cfg2.IsTracingEnabled(), cfg2.IsMetricsEnabled(), @@ -167,12 +167,11 @@ func telemetryPillars() { tracerProvider3 := trace.NewTracerProvider() meterProvider3 := metric.NewMeterProvider() loggerProvider3 := log.NewLoggerProvider() - cfg3 := &otel.Config{ - TracerProvider: tracerProvider3, - MeterProvider: meterProvider3, - LoggerProvider: loggerProvider3, - ServiceName: "full-telemetry-service", - } + cfg3 := otel.NewConfig("full-telemetry-service", + otel.WithTracerProvider(tracerProvider3), + otel.WithMeterProvider(meterProvider3), + otel.WithLoggerProvider(loggerProvider3), + ) fmt.Printf(" Tracing: %v, Metrics: %v, Logging: %v\n", cfg3.IsTracingEnabled(), cfg3.IsMetricsEnabled(), @@ -191,11 +190,10 @@ func configurationValidation() { // Check what's enabled tracerProvider := trace.NewTracerProvider() - cfg := &otel.Config{ - TracerProvider: tracerProvider, - ServiceName: "validation-example", - ServiceVersion: "2.0.0", - } + cfg := otel.NewConfig("validation-example", + otel.WithServiceVersion("2.0.0"), + otel.WithTracerProvider(tracerProvider), + ) fmt.Println("Checking configuration state:") fmt.Printf(" Service Name: %s\n", cfg.ServiceName) diff --git a/examples/ssh/README.md b/examples/ssh/README.md index 0f677aa..413b761 100644 --- a/examples/ssh/README.md +++ b/examples/ssh/README.md @@ -4,7 +4,7 @@ This directory contains examples demonstrating how to use the `ssh` package for ## 📍 Example Code Location -**Full example implementation:** [/ssh/examples/example.go](https://github.com/jasoet/pkg/blob/main/ssh/examples/example.go) +**Full example implementation:** [`example.go`](./example.go) ## 🚀 Quick Reference for LLMs/Coding Agents @@ -49,17 +49,18 @@ The `ssh` package provides utilities for: ## Running the Examples -To run the examples, use the following command from the `ssh/examples` directory: +Run from the repository root. The `example` build tag is required — without it the +file is excluded from the build: ```bash -go run example.go +go run -tags=example ./examples/ssh ``` **Note**: The examples require a working SSH server and appropriate credentials. Update the configuration in the examples to match your environment. ## Example Descriptions -The [example.go](https://github.com/jasoet/pkg/blob/main/ssh/examples/example.go) file demonstrates several use cases: +The [example.go](./example.go) file demonstrates several use cases: ### 1. Basic SSH Tunnel