Google’s HTTP 429 means the request rate or traffic pattern has triggered a temporary limit. Rotating proxies may change one network signal, but they do not fix duplicate queries, uncontrolled retries, poor backoff, or an unsuitable extraction path. A reliable recovery plan starts by reducing pressure, then chooses an API or proxy tier that matches the target and workload.
Quick diagnosis
Before changing IPs, capture the request URL, status code, response headers, retry count, concurrency, and time since the previous attempt. A 429 can be caused by several factors:
- Too many requests from one client or network path.
- Immediate retries that amplify the original overload.
- Repeated or low-value queries that should have been cached.
- A browser-heavy target being requested through a simple HTTP client.
- A provider or proxy tier that does not match the target’s controls.
Treat the response as a signal to slow down and inspect the workload. Do not claim that any proxy makes a request “unblockable”.
A recovery sequence that preserves evidence
1. Stop the retry storm
Pause the affected queue, apply exponential backoff with jitter, and set a maximum attempt count. Do not let every worker retry at the same time. Mark a request as failed after the retry budget is exhausted so the job can be reviewed later.
2. Remove avoidable pressure
Deduplicate identical queries, cache stable results, avoid fetching the same page in parallel, and keep a per-domain concurrency limit. Separate discovery from page extraction so a search burst does not automatically become the same-sized Reader burst.
3. Check the endpoint and request shape
For Google SERP data, use a purpose-built endpoint rather than repeatedly downloading a search results page with a home-grown scraper. SearchCans uses POST https://www.searchcans.com/api/v1/search with t and s in a JSON body.
import time
import requests
response = requests.post(
"https://www.searchcans.com/api/v1/search",
headers={"Authorization": "Bearer YOUR_API_KEY"},
json={
"t": "google",
"s": "web to markdown api",
"country": "us",
"language": "en",
"p": 1,
"d": 30000,
},
timeout=35,
)
if response.status_code == 429:
time.sleep(2.0)
else:
response.raise_for_status()
data = response.json()The sample is intentionally conservative. In production, use a bounded retry helper rather than a fixed sleep and log the server response without storing secrets.
Where proxy rotation fits
Proxy rotation can help when the target blocks a particular network path, but it is not a substitute for correct pacing or permission to access the target. SearchCans Reader has explicit proxy tiers:
proxy |
Tier | Additional credits |
|---|---|---|
| 0 | Direct/default | 0 |
| 1 | Shared Pool | 2 |
| 2 | Datacenter | 5 |
| 3 | Residential | 10 |
Start with proxy: 0 and escalate only when standard mode fails. For a JavaScript-heavy page, use Reader mode: 1; proxy selection and render mode solve different problems.
The Reader endpoint is POST https://www.searchcans.com/api/v1/url. A minimal request looks like this:
reader = requests.post(
"https://www.searchcans.com/api/v1/url",
headers={"Authorization": "Bearer YOUR_API_KEY"},
json={
"t": "url",
"s": "https://example.com/article",
"mode": 1,
"w": 1000,
"d": 30000,
"proxy": 0,
},
timeout=35,
)
reader.raise_for_status()
markdown = reader.json()["data"]["markdown"]Why a managed SERP API can be safer
A managed API gives the application a contract for authentication, request fields, response handling, and account-level concurrency. It does not remove the need for responsible traffic management, and it cannot guarantee that every target will accept every request. It can, however, keep search discovery separate from a DIY proxy pool and make failures easier to measure.
For the broader 429 workflow, see how to fix 429 scraping errors with bounded concurrency.
Monitoring after recovery
Track 429 rate, successful response rate, p95 latency, retries per successful result, duplicate-query rate, and credits per accepted result. Alert on a change in the 429 rate rather than waiting for the entire queue to fail. Keep the original request context so operators can tell whether the cause was a target response, an API response, or client-side overload.
Frequently Asked Questions
Q: Will rotating proxies permanently fix Google 429 errors?
A: No. Rotation may change one network signal, but repeated queries, aggressive retries, browser fingerprints, and target policies can still trigger limits. Fix the request pattern first.
Q: Which SearchCans proxy tier should I use first?
A: Start with proxy: 0. Escalate to a paid tier only when the target requires it, and keep the decision visible in logs because proxy tiers consume additional credits.
Q: Does Reader mode 1 replace a proxy?
A: No. mode: 1 enables headless browser rendering for JavaScript-heavy pages. The proxy field controls the network path. They address different failure modes.
Conclusion
The durable answer to Google 429 errors is controlled traffic, useful retries, and an extraction path that matches the target. Use proxy rotation as one measured option, not as a promise to evade every limit. That keeps the pipeline easier to operate and the data quality easier to defend.
