An async Python client for the seven MrScraper services.
Seven services share the same descriptions, parameter meanings, and defaults across the CLI, SDKs, and integrations. Names, types, authentication, and code examples follow each surface's conventions. See the API Reference.
| Service | Public name | Description |
|---|---|---|
| Status | status |
Return the raw account status API response without summarizing, redacting body fields, or adding calculated fields. |
| Scrape HTML | scrape_html |
Retrieve the raw HTML of a page. Browser rendering is optional. |
| Scrape Markdown | scrape_markdown |
Retrieve a page as Markdown. Including HTML also enables browser rendering and returns both outputs as JSON. |
| Get screenshot | get_screenshot |
Capture a base64 screenshot of the full page or top viewport. Browser rendering is always enabled. |
| Scrape brand | scrape_brand |
Extract a website's colors, typography, logos, and social links as JSON. |
| Scrape JSON | scrape_json |
Extract structured JSON from a page using a natural-language prompt. Append Json Schema: and a JSON schema to shape the output. |
| Web search | web_search |
Search the web and return ranked results with titles, URLs, descriptions, and sources. |
Requires Python 3.9 or newer.
python -m pip install --upgrade mrscraper-sdkObtain an API key from the MrScraper app. API key used to authenticate the request.
export MRSCRAPER_API_TOKEN="YOUR_TOKEN"Use MrScraper() to read the environment or MrScraper(token="YOUR_TOKEN") for an explicit credential. Precedence is a nonempty token, MRSCRAPER_API_TOKEN, then MRSCRAPER_API_KEY. Authentication belongs to the client, not individual service parameters.
import asyncio
from mrscraper import MrScraper
async def main():
client = MrScraper()
result = await client.scrape_html('https://example.com', browser_rendering=True, warm_start=True, wait_for_selector='body', timeout=120, max_retries=0)
print(result["data"])
asyncio.run(main())Examples use example.com as the sample target. Replace it for your use case. Live scraping and search calls consume account quota.
Response examples show selected keys and types observed in live API responses. Values are placeholders; page content, base64 images, and arrays are abbreviated. Actual responses may contain additional fields.
HTML, Markdown, screenshots, and JSON extraction share url, timeout, geoCode, proxyCountry, super, waitUntil, proxy, maxRetries, and saveResult (using the naming style shown below). Page defaults are 300 seconds, US geolocation/proxy country, real-device routing off, and dashboard saving off. Omitted retry limits mean no limit. Exact types, defaults, and standard descriptions appear in each service table.
HTML, Markdown, and screenshots additionally accept blockResources, waitForSelector, and returnCookie. JSON extraction does not accept these controls. Browser rendering is optional for HTML, Markdown, and JSON; screenshots always enable it. Markdown with HTML also enables browser rendering.
Return the raw account status API response without summarizing, redacting body fields, or adding calculated fields.
This service takes no service parameters. Authentication is configured separately.
The HTTP deadline is fixed at 600 seconds. It is not a service parameter.
import asyncio
from mrscraper import MrScraper
async def main():
client = MrScraper()
result = await client.status()
print(result["data"])
asyncio.run(main())The API body contains message and data. Account details remain nested in data, including subscriptionItem and scraperAllowance. The response envelope therefore exposes account details at data.data. Sensitive response headers are removed.
Success response
{
"status_code": 200,
"data": {
"message": "Example response",
"data": {
"id": "ACCOUNT_ID",
"tokenLimit": 100000,
"tokenUsage": 2500,
"stripeStatus": "active",
"user": {
"name": "Example user",
"email": "user@example.com",
"isVerified": true
},
"subscriptionItem": {
"name": "Example plan",
"tokenLimit": 100000
},
"scraperAllowance": {
"remaining": 97500,
"plans": [
{
"remaining": 97500
}
]
}
}
},
"headers": {
"content-type": "application/json; charset=utf-8"
}
}Retrieve the raw HTML of a page. Browser rendering is optional.
| Parameter | HTTP field | Type | Default | Description |
|---|---|---|---|---|
url |
url |
str | Required | Absolute HTTP or HTTPS URL of the page to process. |
timeout |
timeout |
int | 300 |
Page load timeout in seconds, from 1 to 300. The HTTP request allows 30 additional seconds. |
geo_code |
geoCode |
str | "us" |
Two-letter country code to load the page from, sent in lowercase. |
proxy_country |
proxyCountry |
str | "us" |
Two-letter country code of the proxy server, sent in lowercase. |
super |
super |
bool | False |
Whether to route the request through real devices. |
wait_until |
waitUntil |
str: domcontentloaded, load, networkidle0, networkidle2 |
"domcontentloaded" |
Navigation completion event to wait for: domcontentloaded, load, networkidle0, or networkidle2. |
proxy |
proxy |
str | None (omitted) |
Custom proxy URL used to load the page. |
max_retries |
maxRetries |
int | None (omitted) |
Maximum number of retries, as a nonnegative integer. Omit this parameter for no limit. |
save_result |
saveResult |
bool | False |
Whether to save the result to the dashboard Result tab. |
block_resources |
blockResources |
bool | False |
Whether to block images, fonts, and stylesheets while loading the page. |
wait_for_selector |
waitForSelector |
str | None (omitted) |
Wait for the page to contain the specified CSS selector before returning the result. |
return_cookie |
returnCookie |
bool | False |
Whether to include browser session cookies in the response. |
browser_rendering |
browserRendering |
bool | False |
Whether to render the page in a browser before returning the result. |
warm_start |
warmStart |
bool | False |
Whether to enable warm start for more reliable scraping, at the cost of additional latency. |
import asyncio
from mrscraper import MrScraper
async def main():
client = MrScraper()
result = await client.scrape_html('https://example.com', browser_rendering=True, warm_start=True, wait_for_selector='body', timeout=120, max_retries=0)
print(result["data"])
asyncio.run(main())The default response body is HTML text. Additional controls can cause the API to return JSON; the returned structure is preserved.
Success response
{
"status_code": 200,
"data": "<html><body>Example page content...</body></html>",
"headers": {
"content-type": "text/html; charset=utf-8"
}
}Retrieve a page as Markdown. Including HTML also enables browser rendering and returns both outputs as JSON.
| Parameter | HTTP field | Type | Default | Description |
|---|---|---|---|---|
url |
url |
str | Required | Absolute HTTP or HTTPS URL of the page to process. |
timeout |
timeout |
int | 300 |
Page load timeout in seconds, from 1 to 300. The HTTP request allows 30 additional seconds. |
geo_code |
geoCode |
str | "us" |
Two-letter country code to load the page from, sent in lowercase. |
proxy_country |
proxyCountry |
str | "us" |
Two-letter country code of the proxy server, sent in lowercase. |
super |
super |
bool | False |
Whether to route the request through real devices. |
wait_until |
waitUntil |
str: domcontentloaded, load, networkidle0, networkidle2 |
"domcontentloaded" |
Navigation completion event to wait for: domcontentloaded, load, networkidle0, or networkidle2. |
proxy |
proxy |
str | None (omitted) |
Custom proxy URL used to load the page. |
max_retries |
maxRetries |
int | None (omitted) |
Maximum number of retries, as a nonnegative integer. Omit this parameter for no limit. |
save_result |
saveResult |
bool | False |
Whether to save the result to the dashboard Result tab. |
block_resources |
blockResources |
bool | False |
Whether to block images, fonts, and stylesheets while loading the page. |
wait_for_selector |
waitForSelector |
str | None (omitted) |
Wait for the page to contain the specified CSS selector before returning the result. |
return_cookie |
returnCookie |
bool | False |
Whether to include browser session cookies in the response. |
browser_rendering |
browserRendering |
bool | False |
Whether to render the page in a browser before returning the result. |
warm_start |
warmStart |
bool | False |
Whether to enable warm start for more reliable scraping, at the cost of additional latency. |
html |
html |
bool | False |
Whether to include the page HTML in the response. |
import asyncio
from mrscraper import MrScraper
async def main():
client = MrScraper()
result = await client.scrape_markdown('https://example.com', html=True, warm_start=True)
print(result["data"])
asyncio.run(main())The default response body is Markdown text. Enabling HTML returns JSON containing both HTML and Markdown.
With html enabled
{
"status_code": 200,
"data": {
"html": "<html>...</html>",
"markdown": "# Example page\n\n..."
},
"headers": {
"content-type": "application/json"
}
}Default Markdown text
{
"status_code": 200,
"data": "# Example page\n\nExample Markdown content...",
"headers": {
"content-type": "text/markdown; charset=utf-8"
}
}Capture a base64 screenshot of the full page or top viewport. Browser rendering is always enabled.
| Parameter | HTTP field | Type | Default | Description |
|---|---|---|---|---|
url |
url |
str | Required | Absolute HTTP or HTTPS URL of the page to process. |
timeout |
timeout |
int | 300 |
Page load timeout in seconds, from 1 to 300. The HTTP request allows 30 additional seconds. |
geo_code |
geoCode |
str | "us" |
Two-letter country code to load the page from, sent in lowercase. |
proxy_country |
proxyCountry |
str | "us" |
Two-letter country code of the proxy server, sent in lowercase. |
super |
super |
bool | False |
Whether to route the request through real devices. |
wait_until |
waitUntil |
str: domcontentloaded, load, networkidle0, networkidle2 |
"domcontentloaded" |
Navigation completion event to wait for: domcontentloaded, load, networkidle0, or networkidle2. |
proxy |
proxy |
str | None (omitted) |
Custom proxy URL used to load the page. |
max_retries |
maxRetries |
int | None (omitted) |
Maximum number of retries, as a nonnegative integer. Omit this parameter for no limit. |
save_result |
saveResult |
bool | False |
Whether to save the result to the dashboard Result tab. |
block_resources |
blockResources |
bool | False |
Whether to block images, fonts, and stylesheets while loading the page. |
wait_for_selector |
waitForSelector |
str | None (omitted) |
Wait for the page to contain the specified CSS selector before returning the result. |
return_cookie |
returnCookie |
bool | False |
Whether to include browser session cookies in the response. |
screenshot |
screenshot |
str: full, top |
"full" |
Screenshot area to capture: full for the entire page or top for the top viewport. |
html |
html |
bool | True |
Whether to include the page HTML in the response. |
markdown |
markdown |
bool | False |
Whether to include the page Markdown in the response. |
import asyncio
from mrscraper import MrScraper
async def main():
client = MrScraper()
result = await client.get_screenshot('https://example.com', screenshot='top', html=False, markdown=True)
print(result["data"])
asyncio.run(main())The response body contains a base64 screenshot and requested page outputs. The API can still include HTML when HTML output is disabled.
Success response
{
"status_code": 200,
"data": {
"html": "<html>...</html>",
"screenshot": "BASE64_PNG_DATA",
"markdown": "# Example page\n\n..."
},
"headers": {
"content-type": "application/json; charset=utf-8"
}
}Extract a website's colors, typography, logos, and social links as JSON.
| Parameter | HTTP field | Type | Default | Description |
|---|---|---|---|---|
url |
url |
str | Required | Absolute HTTP or HTTPS URL of the page to process. |
The HTTP deadline is fixed at 120 seconds. It is not a service parameter.
The request uses action=brand-scraping and bundle=none; page controls are not accepted.
import asyncio
from mrscraper import MrScraper
async def main():
client = MrScraper()
result = await client.scrape_brand('https://example.com')
print(result["data"])
asyncio.run(main())The API body contains message and a data object with job_id, status, url, final_url, brand, and error. Brand details are available at data.data.brand in the response envelope.
Success response
{
"status_code": 200,
"data": {
"data": {
"job_id": "JOB_ID",
"status": "succeeded",
"url": "https://example.com",
"final_url": "https://example.com",
"brand": {
"name": "Example brand",
"colors": {
"background": "#ffffff",
"text": "#222222"
},
"typography": {
"body": {
"font_family": "Example font"
}
}
},
"error": null,
"bundle_url": null
},
"message": "Example response"
},
"headers": {
"content-type": "application/json"
}
}Extract structured JSON from a page using a natural-language prompt. Append Json Schema: and a JSON schema to shape the output.
| Parameter | HTTP field | Type | Default | Description |
|---|---|---|---|---|
url |
url |
str | Required | Absolute HTTP or HTTPS URL of the page to process. |
timeout |
timeout |
int | 300 |
Page load timeout in seconds, from 1 to 300. The HTTP request allows 30 additional seconds. |
geo_code |
geoCode |
str | "us" |
Two-letter country code to load the page from, sent in lowercase. |
proxy_country |
proxyCountry |
str | "us" |
Two-letter country code of the proxy server, sent in lowercase. |
super |
super |
bool | False |
Whether to route the request through real devices. |
wait_until |
waitUntil |
str: domcontentloaded, load, networkidle0, networkidle2 |
"domcontentloaded" |
Navigation completion event to wait for: domcontentloaded, load, networkidle0, or networkidle2. |
proxy |
proxy |
str | None (omitted) |
Custom proxy URL used to load the page. |
max_retries |
maxRetries |
int | None (omitted) |
Maximum number of retries, as a nonnegative integer. Omit this parameter for no limit. |
save_result |
saveResult |
bool | False |
Whether to save the result to the dashboard Result tab. |
prompt |
prompt |
str | "Extract all data as complete as possible." |
Instructions describing the data to extract. Append Json Schema: and a JSON schema to shape the output. |
browser_rendering |
browserRendering |
bool | False |
Whether to render the page in a browser before returning the result. |
html |
html |
bool | False |
Whether to include the page HTML in the response. |
screenshot |
screenshot |
bool | False |
Whether to include a base64 screenshot in the response. |
The request uses agent=general and GET. If the encoded URL exceeds 8,000 characters, it switches to POST with url, prompt, and agent in the body; page controls remain in the query.
import asyncio
from mrscraper import MrScraper
async def main():
client = MrScraper()
result = await client.scrape_json('https://example.com', prompt='Extract the first 2 items with title and price. Json Schema: {"type":"array","items":{"type":"object","properties":{"title":{"type":"string"},"price":{"type":"string"}},"required":["title","price"]}}', browser_rendering=True)
print(result["data"])
asyncio.run(main())Extraction results are in the API body's data field and follow the requested schema. Response metadata includes thread_id, token_usage, status, error, runtime, and cookies. html, markdown, and screenshot can be empty strings when not requested.
Success response
{
"status_code": 200,
"data": {
"success": true,
"data": [
{
"title": "Example item",
"price": "10.00"
}
],
"thread_id": "THREAD_ID",
"token_usage": 1,
"html": "",
"markdown": "",
"screenshot": "",
"status": "Results ready",
"error": "",
"runtime": 1.23,
"cookies": []
},
"headers": {
"content-type": "application/json"
}
}Search the web and return ranked results with titles, URLs, descriptions, and sources.
| Parameter | HTTP field | Type | Default | Description |
|---|---|---|---|---|
query |
query |
str | Required | Search query text, trimmed before sending, containing 1 to 1000 characters. |
country |
country |
str | "us" |
Two-letter country code to search from, sent in lowercase. |
num_results |
numResults |
int | 10 |
Number of search results to request, as an integer from 10 to 100. Fewer results may be returned. |
The HTTP deadline is fixed at 120 seconds. It is not a service parameter.
import asyncio
from mrscraper import MrScraper
async def main():
client = MrScraper()
result = await client.web_search('site:example.com documentation', country='id', num_results=20)
print(result["data"])
asyncio.run(main())The API body contains success, message, a data array, and tokenUsage. Each result includes rank, title, url, description, and source.
Success response
{
"status_code": 200,
"data": {
"success": true,
"message": "Example response",
"data": [
{
"rank": 1,
"title": "Example result",
"url": "https://example.com",
"description": "Example result description.",
"source": "example.com"
}
],
"tokenUsage": 1
},
"headers": {
"content-type": "application/json; charset=utf-8"
}
}Successful services use an envelope containing status_code, data, and headers. The data field contains the API response body; its field names are not converted to the SDK's naming style. Status preserves the complete body without adding summary fields.
{
"status_code": 200,
"data": "<html><body>Example page content...</body></html>",
"headers": {
"content-type": "text/html; charset=utf-8"
}
}Responses are Python dictionaries. JSON becomes Python values, text remains str, and binary remains bytes. Sensitive response headers are removed for status.
Authentication failures raise AuthenticationError; other HTTP error statuses raise APIError; connection and timeout failures raise NetworkError. APIError.status_code contains the HTTP status when available. Invalid arguments raise ValueError or TypeError before the request.
import asyncio
from mrscraper import MrScraper, APIError, AuthenticationError, NetworkError
async def main():
try:
await MrScraper().scrape_html("https://example.com")
except AuthenticationError as error:
print("Check authentication:", error)
except APIError as error:
print(error.status_code, error)
except NetworkError as error:
print("Request failed:", error)
asyncio.run(main())All methods are coroutines and must be awaited. Examples use asyncio.run(main()); inside a notebook with an active event loop, use await main() instead. Keyword dictionaries can be unpacked with **payload. Optional controls set to None are omitted; explicit False and 0 are preserved.
Reuse an HTTP connection pool by passing http_client=httpx.AsyncClient(...). You manage the supplied client's lifetime; the SDK creates and closes its own client when one is not supplied.
import asyncio
import httpx
from mrscraper import MrScraper
async def main():
async with httpx.AsyncClient() as http_client:
client = MrScraper(http_client=http_client)
print((await client.status())["data"])
print((await client.scrape_markdown("https://example.com"))["data"])
asyncio.run(main())The SDK applies each service's HTTP deadline even when a shared client is supplied.
Check token precedence when authentication fails. Use snake_case argument names rather than HTTP camelCase names. Page timeouts must be integers from 1 to 300; search does not accept a timeout argument. Inspect the raw API body before selecting nested fields.
Replace client.account() with client.status() and read the raw API body. Remove status/search timeout arguments. Replace scraper_api with scrape_html, scrape_markdown, or get_screenshot. Replace full_page with screenshot="full" or "top". Search uses query, country, and num_results. Legacy job, sitemap, rerun, result, and bundle-download methods are outside this interface.
The automated suite checks service behavior without production API calls.
python -m pip install -e . pytest pytest-asyncio httpx ruff mypy
python -m pytest -m 'not integration' -q
ruff check src tests
mypy srcThe following commands call the live API using your credentials and consume quota.
python file.py status
python file.py scrape-html --url https://example.com
python file.py web-search --query "site:example.com documentation"MIT. See LICENSE.
