Extracting SERP data means preserving the requested search context and validating the fields the response actually contains. An organic result, an optional question block and an absent section are different outcomes. Keep those distinctions in the output rather than turn every missing value into an empty successful result.
Freeze the request context before selecting fields
For SearchCans Google results, use POST /api/v1/search with t:"google" and the query in s. Record the country, language, page controls and supported feature switches actually requested. Google Search API contract documents those inputs. A localized query phrase is not a substitute for recording explicit country and language settings.
Validate the organic array and retain missing values
The envelope’s data is an object. Organic results are its organic array, and destination URLs use link. The pure function below accepts a saved response; it makes no network request. It records malformed row positions while preserving missing title, position and snippet values as unknown.
def organic_rows(envelope):
if not isinstance(envelope, dict) or envelope.get("code") != 0:
raise ValueError("Search business response failed")
data = envelope.get("data")
if not isinstance(data, dict) or not isinstance(data.get("organic"), list):
raise ValueError("Expected data.organic array")
rows, rejected = [], []
for number, row in enumerate(data["organic"]):
if not isinstance(row, dict) or not isinstance(row.get("link"), str):
rejected.append(number)
continue
rows.append({"position": row.get("position"), "title": row.get("title"),
"link": row["link"], "snippet": row.get("snippet")})
return {"organic": rows, "rejected_row_indices": rejected,
"available_data_keys": sorted(data)}
The local fixture has one result with no snippet and one malformed row. The accepted result keeps a null snippet and the malformed row is listed separately. A valid empty organic array remains different from a missing array or business failure, which raises an error.
Handle optional SERP blocks explicitly
Request only documented feature controls and inspect the returned keys for that response. Do not guess a People Also Ask field name from another provider’s schema or promise that a section will appear for every query. Store the raw envelope and distinguish “not requested,” “not returned” and “returned empty” only when the request and documented contract support that distinction.
For the question-specific task, use People Also Ask extraction. For repeated domain positions, use rank tracking. For a complete first Python call, use the Python Search API walkthrough. This page owns schema acceptance and missing-field handling, rather than a second general API overview.
