feat(crawl): read the discovery strategies a crawl ran with, and save a page as a file

[zenrows-dev-pool]

Summary

A crawl reports discovery, the strategies it found URLs with, and a create
can ask for them. Reading a page takes download=True to get it as a file to
save. CrawlErrorCode gains api_key_cap_reached. The client dropped all
three, so a caller could not see or set what the API already accepts.

What the task asked for

The client had drifted from the API it wraps. Three things the API serves were
missing here. The discovery parameter and the field echoing it, the
download query parameter on a page read, and one run error code. Markdown
output needed nothing: this client already had it.

How the check confirmed it

uv run ruff check, uv run ruff format --check, uv run ty check src and
uv run pytest all pass, 249 tests. This change adds three checks to the Crawl
acceptance suite: 22 for discovery, 23 for content(download=True), 24 for
the new error code.

A running Crawl service answered the same way. A create with
discovery=["links"] came back with ["links"], and so did a create that left
it out. A create with output_format="markdown" came back with markdown. A
response carrying no discovery at all still parsed.

Encoded by: acceptance checks 22, 23 and 24.

How to try it

crawl = client.create("https://example.com/", discovery=["links"], max_items=5)
print([d.value for d in crawl.discovery])          # ['links']
page = client.content(crawl.crawl_id, result, download=True)

discovery reads ['links'] whether or not you pass it. download=True
returns the same text, asked for as a file to save.

Provenance

discovery is optional on the response model with a default of ["links"],
not required. The API documents that default. An optional field also parses an
answer from a server that does not send it yet. A required one would reject the
whole response instead.

DiscoveryStrategy lists links only, and is open like every other enum here,
so a value this version does not list keeps its raw string.

The lock records the API description this surface follows. It lists
GET /healthz and POST /internal/v1/crawls/{crawl_id}/stop as pending. Both
are in the description and callers cannot reach either. Listing them keeps a
change to either from reading as drift here.

Review

Severity	File and line	Summary
Medium	models.py:107	A required discovery broke 16 tests and would reject any response without the field. Fixed before commit: the field is optional with the documented default.

One pass, by the author. Human must verify: that the default of ["links"] is
the right reading for a crawl created before discovery existed.

Deviations

None.

Record owed

None.