SND (Sunderland) scraper failing — cert fix identified but CMIS server returning 503

[symroe]

Error (2026-06-16)

exceptions.ConnectionError: wreq::Error { kind: Connect, source: CERTIFICATE_VERIFY_FAILED }, uri: https://committees.sunderland.gov.uk/committees/cmis5/Members.aspx

Investigation

Two layered issues found:

1. Certificate not trusted by wreq's BoringSSL

https://committees.sunderland.gov.uk has a TLS certificate that wreq's embedded BoringSSL CA bundle does not trust. Locally, the scraper fails immediately with CERTIFICATE_VERIFY_FAILED.

Fix (verified to clear this layer): add verify_requests = False to the Scraper class in scrapers/SND-sunderland/councillors.py.

2. Server returning 503 after the cert layer is bypassed

After applying verify_requests = False locally, the request gets past the TLS handshake but the CMIS server itself returns:

exceptions.StatusError: wreq::Error { kind: Status(503, None) }

Confirmed independently with plain curl -k, which also returns HTTP 503 with an IIS "Service Unavailable" body, repeatedly over several minutes. This is a server-side outage, not a code or certificate problem.

Fix patterns ruled out

1. HTTPS migration — already HTTPS
2. verify_requests = False — fixes the cert layer (verified), but exposes the underlying 503
3. http_lib = "playwright" — 503 response from IIS; changing HTTP client won't help
4. URL changes — no evidence the CMIS path has moved

What needs to happen

Once committees.sunderland.gov.uk stops returning 503, apply and verify:

class Scraper(CMISCouncillorScraper):
    verify_requests = False

I did not open a PR for this because I can't verify real councillor data comes back while the server is down — happy to revisit once it recovers.

[symroe]

2026-06-17 update: Server is back up (HTTP 200 from committees.sunderland.gov.uk). Applied fix — verify_requests = False clears the cert error and the scraper now runs cleanly with 75 councillors, 74 emails, 75 photos. PR #360 raised.

---

Generated by Claude Code