diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..43dee93 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,38 @@ +name: CI + +on: + push: + branches: [main] + pull_request: + +permissions: + contents: read + +jobs: + lint: + runs-on: ubuntu-24.04 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0 + with: + version: "0.13.1" + + - name: Install pinned tools + run: uv sync --locked + + - name: Ruff lint + run: uv run --locked ruff check + + - name: Ruff format + run: uv run --locked ruff format --check + + - name: ShellCheck scripts + run: git ls-files -z '*.sh' '*.zsh' | xargs -0 uv run --locked shellcheck -s bash + + - name: ShellCheck the status line script embedded in the prompt + run: | + awk '/^```bash$/{f=1;next} /^```$/{f=0} f' claude-code/status-line-prompt.md > "$RUNNER_TEMP/statusline.sh" + uv run --locked shellcheck "$RUNNER_TEMP/statusline.sh" diff --git a/.gitignore b/.gitignore index 6299786..2c62381 100644 --- a/.gitignore +++ b/.gitignore @@ -1,5 +1,22 @@ -# Default ignored files -/shelf/ -/workspace.xml +# Editors .idea/ .vscode/ + +# macOS +.DS_Store + +# Python and uv +__pycache__/ +.venv/ +.ruff_cache/ + +# Files the scripts write when run from inside the repo. They hold personal +# data (alias recipients, DNS records), so they must never be committed. +/*.json +/*.txt +*.usage.txt +*.before-delete.txt +*.scan.txt +forwardemail/*.json +cloudflare/*.json +cloudflare/*.txt diff --git a/.python-version b/.python-version new file mode 100644 index 0000000..f02113f --- /dev/null +++ b/.python-version @@ -0,0 +1 @@ +3.15.0 diff --git a/README.md b/README.md index 9b57cdf..fb1627e 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,37 @@ # tools -Random tools that can be useful in some situations. +Random tools that can be useful in some situations: small command-line utilities, one folder per +service, plus a few write-ups. -## CloudFlare +## Cloudflare -Some tools for CloudFlare's services. +Tools for [Cloudflare](https://www.cloudflare.com)'s services. See [cloudflare/README.md](cloudflare/README.md). -## ForwardEmail +- [zonetool.py](cloudflare/zonetool.py) — back up and restore a zone's DNS records (BIND export/import, + JSON snapshot/restore, and a delete-everything command that backs up first). +- [flush_cache.sh](cloudflare/flush_cache.sh) — purge a zone's whole CDN cache. +- [ddns.sh](cloudflare/ddns.sh) — point an A record at this machine's current public IP (dynamic DNS). -These are some tools I found useful for [ForwardEmail](https://forwardemail.com)'s -services. They have great API documentation [here](https://forwardemail.net/en/email-api). +## Forward Email + +Tools for [Forward Email](https://forwardemail.net)'s services. They have great API documentation +[here](https://forwardemail.net/en/email-api). See [forwardemail/README.md](forwardemail/README.md). + +- [fe-full.py](forwardemail/fe-full.py) — back up a domain's alias settings to JSON and restore them. +- [check-usage.py](forwardemail/check-usage.py) — report the storage each alias uses. +- [backup_emails.sh](forwardemail/backup_emails.sh), [check_usage.sh](forwardemail/check_usage.sh) — run + the two scripts above for a list of domains. +- [reset_pop.sh](forwardemail/reset_pop.sh) — empty a POP3-only mailbox. + +## DNS + +Provider-agnostic DNS tools. See [dns/README.md](dns/README.md). + +- [scandns.py](dns/scandns.py) — rebuild a BIND zone file from DNS answers, for hosts with no export. +- [flatten-spf.py](dns/flatten-spf.py) — flatten an SPF record into IP ranges, and check it for drift. +- [check_email_dns.sh](dns/check_email_dns.sh) — show a domain's MX, SPF, DKIM, DMARC, BIMI, MTA-STS, and + TLS-RPT records. +- [rdap-lookup.zsh](dns/rdap-lookup.zsh) — look up domain registration data over RDAP. ## Claude Code @@ -17,8 +39,12 @@ Configuration for [Claude Code](https://claude.com/claude-code). [claude-telemetry.zsh](claude-code/claude-telemetry.zsh) — environment variables that disable Claude Code's non-essential network traffic (usage analytics, error reporting, feature flags, -auto-updates, and the transcript-uploading `/bug` and `/feedback` commands). Source it from your -shell profile. +auto-updates, and the transcript-uploading `/feedback` command). Source it from your shell profile. +Side effects: Claude Code stops updating itself, so run `claude update` yourself, and Remote Control +is unavailable while feature flags are off. To undo, unset the variables; setting some of them to `0` +still opts out. Shell-profile exports only reach sessions started from a shell that loaded them; the +same variables can go in the `env` block of `~/.claude/settings.json`, which applies however Claude +Code is launched. [status-line-prompt.md](claude-code/status-line-prompt.md) — prompt for generating the status line script. @@ -29,4 +55,79 @@ Write-ups on engineering practice rather than code. [Small-PRs.md](best-practices/Small-PRs.md) — the case for small pull requests: why review quality and merge latency degrade with size, and the patterns for splitting work that *feels* atomic -(refactor-then-behave, vertical slices, expand/migrate/contract, stacked PRs, and dark launches). \ No newline at end of file +(refactor-then-behave, vertical slices, expand/migrate/contract, stacked PRs, and dark launches). + +## Prerequisites + +- macOS or Linux. Developed on macOS (Apple silicon); CI runs on Ubuntu 24.04 (x86_64). +- [uv](https://docs.astral.sh/uv/) 0.13.1, the version CI pins. uv installs Python 3.15.0 (pinned in + `.python-version`) and the exact dependency versions in `uv.lock`, so no separate Python or pip + setup is needed. +- zsh for the shell scripts (the macOS default shell; `apt install zsh` on Linux). +- `curl` and `jq` for the shell scripts, and `dig` for `dns/check_email_dns.sh` (included with macOS; + `dnsutils` on Debian/Ubuntu). +- API credentials, read from environment variables: `CF_TOKEN` for Cloudflare and `API_KEY` for + Forward Email. Nothing secret belongs in this repo. + +## Installation + +```bash +git clone https://github.com/junxit/tools.git +cd tools +uv sync --locked +``` + +## How to Run + +Run the Python tools through uv so they use the pinned environment; the shell scripts run directly. +Each folder's README has the details. + +```bash +export CF_TOKEN=... # Cloudflare API token +uv run cloudflare/zonetool.py example.com export example.com.txt +cloudflare/ddns.sh example.com home.example.com + +export API_KEY=... # Forward Email API key +uv run forwardemail/fe-full.py export example.com example.com.json + +uv run dns/flatten-spf.py example.com +dns/check_email_dns.sh example.com +dns/rdap-lookup.zsh example.com +``` + +## How to Test + +There is no automated test suite. CI ([.github/workflows/ci.yml](.github/workflows/ci.yml)) lints +every pull request and push to `main`; run the same checks locally with: + +```bash +uv run --locked ruff check +uv run --locked ruff format --check +git ls-files -z '*.sh' '*.zsh' | xargs -0 uv run --locked shellcheck -s bash +``` + +## How to Delete / Uninstall + +```bash +# Remove Python cache files +find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null +find . -type f -name "*.pyc" -delete +find . -type f -name "*.pyo" -delete + +# Remove the virtual environment and the linter cache +rm -rf .venv .ruff_cache + +# Remove the files the scripts write (backups may hold personal data; keep any you need first) +rm -f ./*.json ./*.txt ./*.usage.txt ./*.before-delete.txt ./*.scan.txt +``` + +To remove everything, delete the cloned directory. + +## Assumptions + +- Credentials come from environment variables. The Forward Email wrapper scripts still contain an + `API_KEY` placeholder to edit; don't commit a real key. +- The scripts write their output files to the current directory. `.gitignore` keeps the usual output + names out of git, because backups contain personal data such as alias recipients. +- The restore commands (`fe-full.py import`, `zonetool.py import`/`restore`/`delete_all`) change live + settings. Try them on a test alias or zone before relying on them. diff --git a/claude-code/claude-telemetry.zsh b/claude-code/claude-telemetry.zsh index 4fac7d5..4e1d1e5 100644 --- a/claude-code/claude-telemetry.zsh +++ b/claude-code/claude-telemetry.zsh @@ -8,11 +8,20 @@ # Scope note: none of this stops prompts, file contents, or command output # reaching Anthropic — that is the model API itself. Whether conversations # are used for training is an account-level Privacy setting, not an env var. +# Per the docs, the master switch also leaves the WebFetch domain safety +# check in place. +# +# Side effects: Claude Code stops updating itself (`claude update` still +# works), and with feature flags off, Remote Control is unavailable. +# +# Undoing: unset the variables. Setting CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC, +# DISABLE_TELEMETRY, or DISABLE_ERROR_REPORTING to 0 or false still opts out. # Master switch. Suppresses all traffic Anthropic classifies as -# non-essential: the auto-updater, error reporting, and usage telemetry. -# The individual flags below are redundant with this but explicit, and -# stay correct if the definition of "non-essential" ever narrows. +# non-essential: the auto-updater, error reporting, usage telemetry, and +# feature-flag fetches. The individual flags below are redundant with this +# but explicit, and stay correct if the definition of "non-essential" ever +# narrows. export CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 # Statsig usage analytics — the product-metrics pipeline (which commands @@ -37,14 +46,16 @@ export CLAUDE_CODE_DISABLE_VITALS_EMITTER=1 # Auto-update version checks. Largely moot under a Homebrew-managed # install, but it removes the startup ping to the release endpoint. +# Manual `claude update` still works. export DISABLE_AUTOUPDATER=1 -# Removes /bug, which uploads your conversation transcript to Anthropic. -# Disabling it means the transcript can't be sent by accident. -export DISABLE_BUG_COMMAND=1 - -# Removes /feedback, the other transcript-attaching submission path. +# Removes /feedback, which uploads your conversation transcript to +# Anthropic. Disabling it means the transcript can't be sent by accident. export DISABLE_FEEDBACK_COMMAND=1 +# Older name of DISABLE_FEEDBACK_COMMAND, which Claude Code still accepts; +# kept so versions that only know the old name are covered too. +export DISABLE_BUG_COMMAND=1 + # Suppresses in-product feedback surveys and their responses. export CLAUDE_CODE_DISABLE_FEEDBACK_SURVEY=1 diff --git a/claude-code/status-line-prompt.md b/claude-code/status-line-prompt.md index 701ea94..1fc60ff 100644 --- a/claude-code/status-line-prompt.md +++ b/claude-code/status-line-prompt.md @@ -71,7 +71,7 @@ total_input_fmt=$(format_number "$total_input") total_output_fmt=$(format_number "$total_output") # Build status line -# Format: user@host:path (branch) | Model | Tokens: in/out | Context: X% | Cost: $X.XX | Lines: +N/-N | PR #N (state) | Time: Xm Ys [session] +# Format: user@host:path (branch) | Model | Tokens: in/out | Context: X% | Cost: $X.XX | Lines: +N/-N | PR #N (state) | API time: Xm Ys [session] printf "\033[1;32m%s@%s\033[0m:\033[1;34m%s\033[0m%s \033[0;90m|\033[0m " \ "$user" "$host" "$dir_display" "$git_branch" @@ -104,7 +104,7 @@ if [ -n "$pr_number" ]; then fi fi -printf "\033[1;33mTime:\033[0m \033[0;36m%s\033[0m" "$duration_fmt" +printf "\033[1;33mAPI time:\033[0m \033[0;36m%s\033[0m" "$duration_fmt" if [ -n "$session_name" ]; then printf " \033[0;90m[%s]\033[0m" "$session_name" diff --git a/cloudflare/README.md b/cloudflare/README.md index 33a0b22..7e9ddea 100644 --- a/cloudflare/README.md +++ b/cloudflare/README.md @@ -14,7 +14,42 @@ such. `flush_cache.sh` takes your API token with `purge:cache` permissions and lets you specify your domain on a command line for which you want to purge all cache entries (and force Cloudflare to refresh its cache). -`zonetool.py` is a tool to export and import DNS records from a Cloudflare zone. - It is a quick way to flush/purge the cache without logging in to your Cloudflare portal, especially if you have many -domains that you copy content to from the command line like I do (I use [rsync](https://en.wikipedia.org/wiki/Rsync)) \ No newline at end of file +domains that you copy content to from the command line like I do (I use [rsync](https://en.wikipedia.org/wiki/Rsync)) + +## Back Up and Restore DNS Records + +`zonetool.py` exports and imports the DNS records of a Cloudflare zone. It reads an API token from the `CF_TOKEN` +environment variable: `export` and `list` need Zone:Read and DNS:Read permissions on the zone, while `import`, +`restore`, and `delete_all` need DNS:Edit. + +```bash +export CF_TOKEN=... +uv run cloudflare/zonetool.py example.com export example.com.txt # BIND file: every record, Cloudflare's own format +uv run cloudflare/zonetool.py example.com import example.com.txt # load a BIND file back in +uv run cloudflare/zonetool.py example.com list example.com.json # JSON snapshot, including comments and tags +uv run cloudflare/zonetool.py example.com restore example.com.json # recreate records from a snapshot +uv run cloudflare/zonetool.py example.com delete_all # back up, then delete every record +``` + +- `export` and `import` are the complete round trip. Cloudflare limits imports to 256 KiB files and 3 requests a + minute. +- `restore` recreates the records in a snapshot. When Cloudflare reports that a record already exists, the record with + the same type, name, and value gets the snapshot's TTL, proxy status, comment, and tags. Records that share a name + (several MX or TXT records, say) are matched by value, so the wrong one is never overwritten. +- `delete_all` asks twice, then writes a timestamped `.-