Skip to content

Commit 01d148c

Browse files
author
Claude Code
committed
Fix STV (Stevenage): HTTPS migration and certificate verification bypass
The scraper was using http://democracy.stevenage.gov.uk which causes wreq to attempt an IPv6 (AAAA) DNS lookup that returns ServFail from Stevenage's DNS server, causing a hard failure. Switching to https:// bypasses this path. The https:// endpoint has an untrusted certificate (not in wreq's BoringSSL bundle), so verify_requests = False is also needed to allow the connection. With both changes the ModGov ASMX endpoint returns valid XML: 39 councillors, 39 with photos, 0 emails (council does not publish email addresses via ModGov).
1 parent 8762213 commit 01d148c

2 files changed

Lines changed: 2 additions & 2 deletions

File tree

scrapers/STV-stevenage/councillors.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,4 @@
22

33

44
class Scraper(ModGovCouncillorScraper):
5-
pass
5+
verify_requests = False

scrapers/STV-stevenage/metadata.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
},
1515
"services": {
1616
"councillors": {
17-
"base_url": "http://democracy.stevenage.gov.uk",
17+
"base_url": "https://democracy.stevenage.gov.uk",
1818
"cms_type": "ModernGov"
1919
}
2020
}

0 commit comments

Comments
 (0)