feat(batch): agents could not recover, inspect or manage batch jobs from the CLI

README.md

@@ -128,20 +128,75 @@ zenrows extract <url> --output markdown
 
 Zenrows **Batch** (`https://async.api.zenrows.com/v1`) fans a
 protected fetch/extract out over many URLs. The cloud subcommands work
-with any active API key and spend the same credits as Fetch. Local spec
-validation + credit estimation always work (no key needed).
+with any active API key and cost the same credits as Fetch: Batch is the
+reliable path at scale, at Fetch's prices. Local spec validation + credit
+estimation always work (no key needed).
 
 ```bash
-zenrows batch estimate jobs.jsonl                    # validate the spec + estimate credits (local)
-zenrows batch create jobs.jsonl --wait               # submit a job and wait for it to finish
-zenrows batch status <job-id>                        # run status + progress stats
-zenrows batch results <job-id> --out results.jsonl   # collect results (paginated)
-zenrows batch cancel <job-id>                        # stop an in-flight run
+zenrows batch estimate jobs.jsonl                     # validate the spec + estimate credits (local)
+zenrows batch create jobs.jsonl --follow              # submit a job and wait for it to finish
+zenrows batch list                                    # your jobs, newest first
+zenrows batch status <job-id>                         # run status, progress stats, pause state
+zenrows batch wait <job-id> --timeout 30m             # wait for the run to end
+zenrows batch results <job-id> --out results.jsonl    # collect results (paginated)
+zenrows batch content <job-id> <task-id>              # one task's result body
+zenrows batch pause <job-id>                          # pause the run (resume <job-id> continues it)
+zenrows batch cancel <job-id>                         # stop an in-flight run
+zenrows batch retry-failed <job-id> --include-pending # rerun every task that did not succeed
+zenrows batch rerun <job-id>                          # rerun failed + pending tasks (--all: every task)
+zenrows batch delete <job-id> --yes                   # delete the job, its runs and results
 ```
 
 `jobs.jsonl` is one JSON object per line, each with a `url` (plus optional
 per-line overrides like `js_render`, `premium_proxy`, `proxy_country`,
-`extract`). Validate and estimate locally first, then submit with `create`.
+`extract`, nested `zenrows_params`, an `external_id`, string `metadata`, and
+`"method": "POST"` with a `body` of up to 16 KiB). `estimate` and `create`
+check every line against the API's limits before anything is sent: `url` up
+to 2048 bytes, `external_id` up to 128 letters, digits, `.`, `_` or `-`,
+and `metadata` up to 20 string entries (4 KiB). Validate and estimate locally
+first, then submit with `create`. `estimate` takes `create`'s
+`--manual` / `--js-render` / `--premium-proxy` / `--proxy-country` flags and
+prices the same request, as a range (Adaptive Stealth Mode costs 1 to 25).
+
+`create` also takes `--name`, `--external-id` and `--metadata key=value`
+(repeatable) for the job itself. It sends an `Idempotency-Key` (generated, or
+`--idempotency-key`) and retries network errors and 502/503/504 under it; an
+outcome that stays unknown prints the command that repeats the request with
+that key, which cannot create a second job.
+
+The local policy caps a run at 10,000 URLs and 250,000 estimated credits by
+default (an Adaptive Stealth Mode task counts 25), so a default run of up to
+10,000 URLs passes in any mode. Raise them for larger jobs (the API takes up to
+100,000 tasks per job): `zenrows policy set max_pages_per_run 100000` and
+`zenrows policy set max_credits_per_run 2500000`. A workspace created by an
+earlier version keeps its caps (1,000 and 10,000) in `policy.json`.
+
+`wait`, `create --follow` and `rerun --follow` poll until the run ends, the run
+is paused, or `--timeout` runs out (default 10 minutes; `30s`, `5m`, `1h`; a
+bare number is milliseconds, as before). When the time runs out, or the run is
+paused, the command exits 0 and prints the run with a `note` and the next
+command; the job keeps running. Ctrl-C stops the wait, not the job, and exits
+130. A run that ends `failed` exits 1: out of credits is
+`POLICY_MAX_CREDITS_EXCEEDED`, an inactive subscription `AUTH_INVALID`, a key
+cap `KEY_CREDIT_CAP_REACHED` (the same codes a refused request gets), anything
+else `BATCH_FAILED`, each with the run's `failure_reason` as `server_code` and
+the `job_id`. Then run the unfinished tasks with
+`zenrows batch rerun <job-id> --status failed,pending,processing`. With no
+flag, `rerun` reruns the `failed` and `pending` tasks, so successful tasks are
+never billed again by surprise; `--status <list>` picks the statuses and
+`--all` reruns every task (not with `--status`). A partial rerun works for 10
+days after the run started; `--all` always does.
+
+With `--json`, `create`, `status`, `wait`, `cancel`, `pause`, `resume` and
+`rerun` print `{ok, job_id, run_id, status, stats, job}` (`jobId` is kept for
+older scripts), plus `paused`, `ingest_status`, `accepted_tasks`, `rerun_of`,
+`retried_tasks` and `inherited_tasks` when the API sends them. Results are
+kept 14 days by default and then read `expired`; a `result_url` is valid 2
+hours. Lifecycle errors get their own codes (`BATCH_RUN_IN_PROGRESS`,
+`BATCH_NO_MATCHING_TASKS`, `BATCH_RERUN_WINDOW_EXPIRED`,
+`BATCH_REQUEST_IN_FLIGHT`, `BATCH_RESULT_EXPIRED`, ...) and carry the API's code
+as `server_code`. Schedules, webhooks, ZIP exports, CSV inputs and open jobs
+are available through the Batch API and the SDKs, not this CLI.
 
 **Extract in Batch.** Set `"extract": "auto"` on a line to run that URL through
 Extract — structured data instead of raw HTML:
@@ -186,7 +241,7 @@ zenrows crawl stop <crawl-id>                           # stop a running crawl
 follows links until `--max-items` or `--max-pages` stops it, or nothing is left
 to open. `--max-items` and `--max-pages` cap the crawl (API default 100 each),
 and `--max-pages` bounds the cost, since each page is one fetch. The local policy `max_pages_per_run` caps `--max-pages` (default
-1000). `--output-format html` uses up the page budget: each kept page is one
+10000). `--output-format html` uses up the page budget: each kept page is one
 more fetch, and `--max-pages` counts it. `--include-patterns` /
 `--exclude-patterns` are repeatable substring filters on the URL. A crawl whose
 pages held nothing new to follow ends `completed` with no results and exits 0.

agent-plugin/skills/batch-jobs/SKILL.md

@@ -7,7 +7,8 @@ version: 0.1.0
 # Batch Jobs (MCP)
 
 Process large workloads asynchronously. Batch is where Zenrows' high-scale
-anti-bot advantage becomes obvious — thousands to millions of URLs.
+anti-bot advantage becomes obvious: thousands to 100,000 URLs per job, at
+Fetch's prices.
 
 > Cloud Batch lives on `async.api.zenrows.com/v1`. A 403 maps to
 > `BATCH_ACCESS_DENIED`. Do not fake a successful
@@ -33,10 +34,22 @@ Optional if exposed by the server: `batch_wait` to block until terminal status.
    `autoparse`, `extract`, `external_id`, `metadata`). `extract: "auto"`
    returns structured data instead of raw HTML, at the same credit cost as a
    regular task; the result carries `html` and `parsed` keys.
-3. Poll `batch_status` until terminal (`completed` / `stopped` / …).
-4. Collect with `batch_results`.
+3. Poll `batch_status` until terminal (`completed`, `stopped`, `failed` or
+   `deleted`). A paused run (`pause_state: paused`) is not terminal and picks
+   up no new task until resumed: report it instead of polling on.
+4. Collect with `batch_results`. Results are kept 14 days by default (then a
+   row reads `expired`); a `result_url` is valid 2 hours.
 5. `batch_cancel` if the user aborts or the run should stop early.
 
+A run ends `failed` when an account problem stops it: `failure_reason`
+`insufficient_credits` (add credits), `subscription_inactive` (fix it in
+Billing) or `api_key_cap_reached` (the key's own cap; the account still has
+credits). The tasks not yet run stay `pending`: once fixed, rerun them with
+status `failed,pending,processing` rather than resubmitting the job.
+
+Schedules, webhooks, ZIP exports and CSV inputs are available through the
+Batch API and the SDKs.
+
 ## Rules
 
 - **Never batch before validating** on a single page ([[extract]] /

recipes/batch-estimate/RECIPE.md

@@ -23,11 +23,14 @@ zenrows batch estimate jobs.jsonl
 ```
 
 Swap `jobs.jsonl` for your own spec. Per-line keys (`js_render`,
-`premium_proxy`, `proxy_country`, `mode`, `autoparse`, `external_id`) tune each
-URL; cost multipliers are basic 1x / JS 5x / proxy 10x / both 25x.
+`premium_proxy`, `proxy_country`, `mode`, `autoparse`, `external_id`,
+`metadata`, `method`, `body`) tune each URL; cost multipliers are basic 1x / JS
+5x / proxy 10x / both 25x, and Adaptive Stealth Mode costs 1x to 25x. Pass
+`create`'s job-level flags (`--premium-proxy`, `--js-render`, `--manual`) to
+`estimate` to price the same request.
 
 Once the spec is clean, submit it:
 ```
-zenrows batch create jobs.jsonl --wait
+zenrows batch create jobs.jsonl --follow
 ```
 See [[batch-jobs]] for the full cloud flow.