Check the status code, honor the encoding and reuse the HttpClient when downloading

[capjan]

Summary

The remaining network items from the phase 4 plan, plus the findings of two reviews. Three of the changes alter behavior, please read that part.

Behavior changes

1. Error status codes. DownloadToString returned the content of the error page for a 404 or 500, so TryDownloadToString reported success for a missing resource. It now throws an HttpRequestException, and TryDownloadToString returns false.
2. The real exception instead of an AggregateException. The synchronous wrappers used .Result, which wraps every failure (404 in DefaultDownloader, no connection, ...) in an AggregateException. They use GetAwaiter().GetResult() now, so callers get the HttpRequestException itself, and all downloaders behave the same. Code that catches AggregateException around these calls has to catch HttpRequestException instead.
3. No cookies. The clients that are shared or reused do not store cookies any more:
   • DefaultDownloader and DownloaderWithCredentials: a per-download client is replaced by a shared one, and a cookie container on a shared client would mix the sessions of independent users and jobs. (Review finding, P1: reproduced with a local server, the cookie of one downloader was sent by another.)
   • HttpChannelExt.SharedHttpClient, which the IHttpChannel extension methods use, is a process-wide static client that always had a cookie container, so it had this problem before this PR. It is fixed too. Anyone relying on cookies being kept between channel.DownloadToString calls has to assign a client with UseCookies = true to HttpChannelExt.SharedHttpClient, or use DefaultDownloader(HttpClient) with a client of their own.

If the release should stay free of behavior changes until 13.0, the three points are the ones to hold back; the rest does not depend on them.

Fixes

• DownloadToString ignored its encoding parameter. An explicit encoding is used now, but a byte order mark in the content still wins over it. Without an explicit encoding nothing changes: HttpContent.ReadAsStringAsync uses the charset the server announces (UTF-8, or the encoding of a byte order mark, if there is none) and does not look at a byte order mark when a charset is announced.
• HttpChannelDownloader always passed UTF-8 when no encoding was given. Honoring the parameter alone would have forced UTF-8 for servers that announce another charset, so it passes null now and only an explicit encoding is forced.
• DownloaderWithCredentials requires its credentials (ArgumentNullException).

Performance

• DefaultDownloader and DownloaderWithCredentials created a new HttpClient for every download (socket exhaustion, slow). DefaultDownloader and HttpChannelExt.SharedHttpClient now start with the same client (one connection pool), DownloaderWithCredentials creates its client once per instance.
• New constructor DefaultDownloader(HttpClient) to configure the client (proxy, timeout, cookies, ...). The client is not disposed by the downloader. The parameterless constructor is unchanged.
• The shared clients use a SocketsHttpHandler that recycles connections every two minutes, so a long-lived client notices changed DNS entries. netstandard2.0 has no SocketsHttpHandler, so it uses a plain HttpClientHandler there (first #if NETSTANDARD2_0 in the code base).

The public API only grows (one constructor). The IDownloader documentation describes the behavior.

Not part of this PR

• Async interfaces. They need the decision on parallel IAsync* interfaces.
• DownloadHeader still does not check the status code on purpose: many servers answer HEAD with 405 and the headers are still useful.

Tests

• DownloadToStringTest (fake server): error status, explicit and server encoding, byte order mark (also against an explicit encoding), HttpChannelDownloader with and without an encoding
• DefaultDownloaderTest: the given client is used, keeps its timeout and is not disposed, exceptions are not wrapped, credentials are required, DownloaderWithCredentials sends the credentials and can be used repeatedly (local server with basic authentication)
• DownloaderCookieTest (local server): independent downloaders, repeated downloads, DownloaderWithCredentials and the default IHttpChannel do not share cookies; a client the caller owns may keep them
• LocalHttpServer had a flaw that made about 1 in 20 runs fail: Dispose called Stop() and then Close(). The managed HttpListener then removes the prefix a second time, which fails with "Address already in use" while pooled connections are open. Close() alone is enough.

Verification

• Each new test for a finding failed before its fix and passes after it
• 296/296 tests pass on net10.0 and net8.0 (local, macOS); 200 runs of the local-server tests and 100 full runs without a failure
• Build with analyzers as errors: 0 warnings on all three target frameworks
• Package validation against Cap.Core 12.0.0: nothing removed (only the known ISpanFormattable noise from dropping net6.0)
• The local-server tests have not run on the Windows and macOS CI runners yet with the last commit. If they turn out to be unstable there, they are the first candidates to adjust.