feat(batch): verify webhook signatures, skip expired results and match the API's responses

[emirlan-zen]

What this changes

This PR brings the Batch client in line with what the Batch API returns. It also adds the helpers that the Node and Go SDKs get under the same names. It is additive: no exported name, field, model or signature is removed or narrowed. griffe check zenrows -s src --against origin/main reports nothing.

New public API (all exported from zenrows.batch):

• verify_webhook_signature(raw_body, signature_header, secrets, *, tolerance_seconds=300, now=None) -> bool checks the X-Signature header (t=<unix>,v1=<hex>,kid=<kid>) on a signed delivery.
  • secrets maps each kid to the base64 secret from rotate_hmac_key(). The helper decodes it and picks it by kid, so it keeps working during a key rotation.
  • It computes HMAC-SHA256 over <t>.<raw body> and compares in constant time.
  • It rejects a timestamp more than tolerance_seconds off. 0 turns the check off.
  • A secret that does not decode to exactly 32 bytes never verifies. It returns False on bad input and never raises.
  • It also ships the constants WEBHOOK_SIGNATURE_HEADER, WEBHOOK_EVENT_ID_HEADER and DEFAULT_WEBHOOK_TOLERANCE_SECONDS, and re-exports WebhookEvent / WebhookEventType for the payload.
• BatchErrorCode: a str enum naming every error code the Batch API sends (43 codes). Members compare equal to BatchAPIError.code.
• JobRef.add_tasks(body, *, idempotency_key=None).
  • A repeat with the same key and body returns the first answer instead of adding the tasks twice.
  • With a key, the client retries network errors and 429/502/503/504.
  • Without a key, nothing changes: the call is sent once.
• on_skip: Callable[[TaskResult], None] | None = None on download_to_dir and download_to_memory (on a run and on job.run).
• RunPaused(WaiterError) with .run, and wait_while_paused: bool = False on wait_for_run, run.wait() and job.run.wait().
• wait_for_ingest: bool = False on JobRef.rerun and JobRef.retry_failed.
  • It polls the run the call returned (GET /jobs/{id}/runs/{run_id}, up to 300 s) until its ingest_status is no longer pending, and returns that fresh snapshot.
  • It follows the returned run even when an idempotent replay answers with an older run while a newer one is the job's latest.
  • It does not stop at a terminal status. A run stopped mid-ingest stays ingest_status: pending until ingestion winds down, and it comes back stopped with only the rows written so far.
  • submit_scheduled has no wait option: creating a scheduled job starts no run (the API answers 201 with no latest_run), so there is nothing to ingest until the schedule fires.
• TaskStatus.EXPIRED, and Run.trigger typed as RunTrigger (manual / scheduled). The API already sends both.

Behaviour changes:

• Bulk downloads no longer stop on an expired or failed row.
  • After the retention window (14 days by default), a successful task reads as expired and has no result_url. It still matches status="successful".
  • download_to_dir and download_to_memory used to raise ValueError on the first such row, part-way through.
  • They now skip it, pass it to on_skip, and log one WARNING on zenrows.batch.download with the counts per status. Skipped rows do not count toward max_files / max_count.
  • A successful row without a result_url still raises, so a result never goes missing silently.
  • The single-task helpers raise a clearer ValueError for an expired task.
• Run waiters no longer spin on a paused run.
  • A paused run never progresses until someone resumes it.
  • The waiter used to poll it until its timeout. It now raises RunPaused on the first poll that sees pause_state: paused while the run is not yet in a target status.
  • RunPaused subclasses WaiterError, so an existing except WaiterError still catches it.
  • A paused run that is already terminal (for example stopped) is returned as before.
• A 429 quota_exceeded is no longer retried. It is an account limit (active jobs, scheduled jobs, or an open job's task cap) that clears only when a job finishes, is closed or is deleted. Other 429s are retried as before.
• estimate_cost reads flags the way the API does.
  • js_render / premium_proxy count as on only for True or the string "true" in any case.
  • mode counts as auto only for exactly "auto".
  • The API rejects 1, "1", "yes" and "on" with a 400. The estimator used to price those as on, for jobs the API would refuse.
• Waiting for a submit's ingestion follows the right run.
  • JobRef.wait_for_ingest() and submit_job(..., wait_for_ingest=True) now poll the run the submit returned, by id, when the ref has one. A replayed submit therefore never follows a newer run.
  • A ref without a submit response still polls the job's latest run.
  • The wait now ends only when ingest_status is no longer pending. A run stopped mid-ingest used to end it at once, before its rows and stats.total were settled.
• TaskResult.spend and TaskSpend are kept but documented as deprecated. Result rows never carry a per-task spend, so the field is always None. Read run.stats.spend instead.

Spec refresh (docs/openapi.yaml, models regenerated):

• Add-tasks takes an Idempotency-Key.
• The task list limits are 100,000 per submit and 10,000 per add-tasks call. This widens the request models' max_length.
• Submit and add-tasks list the 429 quota_exceeded response.
• The stale "scheduled returns 501" note is removed.
• A result_url is valid for 2 hours.
• The export cap is 100 GiB.

Docs, docstrings and examples:

• The export cap is 100 GiB, not 1 GiB, and the failure message reads "results are larger than N gb".
• Result links last 2 hours and results are kept 14 days.
• Webhook events: one per run, none on stop, retried for up to 3 days, deduplicated on X-ZenRows-Event-Id. There is a receiver example.
• A partial rerun works within 10 days. Every rerun is accepted asynchronously, so its results can be partial until ingest finishes.
• Target 404/410 responses are billed.
• extract_fields works in Batch.
• The waiter's behaviour on a paused run is documented.
• Example 09's one-shot At date is computed in the future. Example 07 explains the base64 secret. Example 06 handles partial_rerun_window_expired. Example 02 reports skipped rows.
• docs/batch-client-reference.md is regenerated and gains a "Webhook signatures" section.

How it was tested

• make check (ty, ruff, ruff format) and make test: 295 passed, 1 deselected (e2e).
• griffe check zenrows -s src --against origin/main: no breaking changes.
• New tests:
  • tests/test_webhooks.py pins verify_webhook_signature to a fixed vector produced by the API's signing routine (secret bytes 0x00..0x1f, t=1790000000). It covers using the base64 text as the key, tampering, rotation by kid, tolerance bounds, malformed headers and bad secrets.
  • tests/test_batch_client.py covers:
    • expired and failed rows skipped (serial and concurrent, with caps)
    • a successful row without a URL still raising
    • the add-tasks key being sent and reused across a 503 retry
    • no retry without a key
    • no retry on quota_exceeded while a plain 429 is still retried
    • BatchErrorCode against the full code list
    • RunPaused on a paused run, for a specific run and for job.run
    • a paused but terminal run being returned
    • wait_while_paused=True polling through to completion
    • wait_for_ingest=True on rerun and retry_failed polling the returned run until ingest is done
    • the wait following a replayed older run and not stopping at a run stopped mid-ingest
    • no poll without the flag
    • submit_scheduled's real 201 with no run returning after one request
    • the job-level wait not stopping at a run stopped mid-ingest
    • a waited submit polling the submitted run by id while a newer run is the job's latest
  • tests/test_estimate.py covers the accepted and rejected flag spellings and the exact mode: "auto".
• The README Python blocks and every example compile.
• make generate and make docs were re-run, and their output is committed.