SERP Data JSON: Parse API Results in Python

Quick answer

SERP data JSON is the machine-readable form of a search results page. With SearchCans, a JSON request can return organic results plus optional sections such as Knowledge Graph, People Also Ask, news, videos, and related searches. In Python, parse each section defensively because optional SERP features can be absent for a query.

If you are building a rank tracker, research workflow, RAG pipeline, or AI agent, the useful model is simple: send a query, keep the raw response, normalize the fields your application needs, and treat every optional section as nullable. This keeps downstream code stable when two searches produce different SERP layouts.

Start with the response contract

The first mistake in a SERP integration is to treat a search page as one flat list. A real result can contain organic links, an entity panel, questions, news, video results, and related queries. A JSON response keeps those regions separate so your application can decide what to use.

SearchCans uses a single POST endpoint for Google search requests:

https://www.searchcans.com/api/v1/search

The request is authenticated with a Bearer token. A minimal response has a top-level status code and a data object. The exact response can vary by query, but the important shape looks like this:

{
  "code": 0,
  "data": {
    "organic": [
      {
        "page": 1,
        "position": 1,
        "title": "Example result",
        "link": "https://example.com/page",
        "snippet": "A short result description.",
        "displayed_link": "https://example.com"
      }
    ],
    "knowledgeGraph": {},
    "peopleAlsoAsk": [],
    "relatedSearches": []
  }
}

Do not assume that an empty object or array means the request failed. It may simply mean that Google did not show that SERP feature for the query. Check the response status first, then read each section independently.

Send a JSON request from Python

The endpoint accepts a JSON body. There is no SDK requirement for this integration, so a normal Python HTTP client is enough.

import os
import requests

endpoint = "https://www.searchcans.com/api/v1/search"
api_key = os.environ["SEARCHCANS_API_KEY"]

payload = {
    "t": "google",
    "s": "search api with markdown output",
    "country": "us",
    "language": "en",
    "p": 1,
    "knowledgeGraph": True,
    "peopleAlsoAsk": True,
    "peopleAlsoSearchFor": True,
}

response = requests.post(
    endpoint,
    headers={
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json",
    },
    json=payload,
    timeout=30,
)
response.raise_for_status()

body = response.json()
if body.get("code") != 0:
    raise RuntimeError(f"Search request failed: {body.get('code')}")

data = body.get("data") or {}
for result in data.get("organic") or []:
    print(result.get("position"), result.get("title"), result.get("link"))

Keep the API key in an environment variable or a secret manager. Do not place it in a notebook, a committed source file, a browser bundle, or an article example.

Map the main JSON fields

The organic array is usually the starting point for SEO and research applications. Each item represents one organic result, not one complete web page.

The difference between link and displayed_link matters. Fetching should use link. The displayed value may be shortened or formatted for presentation. If you are building a rank tracker, store the target URL separately from the visible display URL and normalize redirects before comparing pages.

Read optional SERP features safely

SearchCans exposes optional fields for parts of the SERP beyond the blue-link list. The Google Search API product page documents knowledgeGraph, peopleAlsoAsk, topStories, inlineVideos, and relatedSearches as separate response areas.

def optional_list(data, name):
    value = data.get(name)
    return value if isinstance(value, list) else []


def optional_object(data, name):
    value = data.get(name)
    return value if isinstance(value, dict) else None


questions = optional_list(data, "peopleAlsoAsk")
related_queries = optional_list(data, "relatedSearches")
news = optional_list(data, "topStories")
videos = optional_list(data, "inlineVideos")
entity = optional_object(data, "knowledgeGraph")

for question in questions:
    print(question.get("question", ""), question.get("snippet", ""))

if entity:
    print("Entity:", entity.get("title", ""))

This pattern avoids a common production bug: calling .get() on None because a query did not produce a Knowledge Graph panel. It also makes it clear that a missing feature is a valid search result, not necessarily an API error.

AI Overview and raw SERP HTML

The request supports aiSummary and html. These options should not be treated as interchangeable. The product documentation describes aiSummary as an option for extracting AI Overview data. Setting html to 1 also returns the raw SERP HTML, which can contain the complete rendered structure of the page.

Do not hard-code an aiOverview object that you expect to exist on every response. If your application needs the full markup for an analysis step, request the raw HTML deliberately and keep that path separate from the structured JSON parser.

Handle pagination and geo targeting

SearchCans supports two pagination styles:

• p requests a specific SERP page, such as page 2.
• page requests a batch from page 1 through a specified page number.

The current product documentation warns against setting both values above 1 in the same request. When both are greater than 1, p takes priority. Pick one mode per request and record it with the response so later rank comparisons are not ambiguous.

page_two = {
    "t": "google",
    "s": "google search api",
    "country": "us",
    "language": "en",
    "p": 2,
}

pages_one_to_three = {
    "t": "google",
    "s": "google search api",
    "country": "us",
    "language": "en",
    "page": 3,
}

Use country and language together when you need a reproducible regional view. A result observed for country: "us" and language: "en" is not automatically the result a user will see in another country or language. Store both values with the query, along with the collection time.

Normalize results for an application

A downstream database should keep the raw response and a smaller normalized record. The raw response protects you when you later need a field that was not part of the first schema. The normalized record makes common queries fast.

from datetime import datetime, timezone


def normalize_results(body, query, country, language):
    data = body.get("data") or {}
    collected_at = datetime.now(timezone.utc).isoformat()
    normalized = []

    for item in data.get("organic") or []:
        normalized.append(
            {
                "query": query,
                "country": country,
                "language": language,
                "page": item.get("page", 1),
                "position": item.get("position"),
                "title": item.get("title", ""),
                "url": item.get("link", ""),
                "snippet": item.get("snippet", ""),
                "collected_at": collected_at,
            }
        )

    return normalized

For rank tracking, compare a normalized URL to the target URL after applying your own canonical URL policy. For content research, use the organic links as discovery inputs, then read selected URLs with a content extraction step. A SERP result tells you which page Google surfaced. It does not contain the full article body.

JSON versus raw HTML

Use structured JSON when your application needs predictable fields such as title, URL, position, snippets, questions, or related queries. It is the better starting point for rank tracking, search monitoring, keyword research, and agent tool calls.

Use raw HTML only when the structured response does not contain the visual or markup detail you need. HTML parsing is more fragile because layout and class names can change. It also creates more work for sanitization and storage.

For an AI research workflow, a practical sequence is:

1. Query the SERP API for discovery and structured result data.
2. Select relevant result URLs instead of sending every result into the next step.
3. Use the Reader API to extract the selected pages into clean Markdown.
4. Keep the query, URL, collection time, and source metadata with the extracted context.
5. Ask the model to distinguish the SERP observation from claims made by the source page.

This separation gives an agent both the search context and the source content without pretending that a snippet is evidence for the whole page.

Common parsing mistakes

Assuming every feature exists

Organic results may exist without a Knowledge Graph, PAA block, news carousel, or video section. Use empty-list and nullable-object handling.

Mixing page and position

Position 1 on page 2 is not the same observation as position 1 on page 1. Store the page number and define whether your application reports page-local position or an absolute position.

Treating snippets as page content

Snippets are search presentation data. They can be truncated, rewritten, or absent. Fetch and read the destination page when the workflow requires evidence or full context.

Hiding the query context

The same URL can rank differently by country, language, page number, and collection time. Store those request parameters with each result.

Hard-coding old field names

Use the current SearchCans documentation and a real response sample as the contract. Do not silently convert current fields such as organic or peopleAlsoAsk into undocumented names like organic_results or people_also_ask.

Putting credentials in client-side code

SERP requests belong on a controlled server or trusted backend job. A public browser bundle would expose the Bearer token and allow other users to spend the account credits.

Build from the official examples

The Google Search API documentation contains the current endpoint, request parameters, Python and cURL examples, pagination behavior, response sections, and product CTA. The API Playground is useful for inspecting a request interactively before writing a parser.

If the next step is to fetch and clean the pages found in a SERP, continue with the Reader API. For a more complete Python request flow, see the Google Search API in Python guide.

Frequently asked questions

Q: What does a SERP API JSON response contain?

A: It normally contains a response status and a data object. The data can include an organic result array and optional sections such as Knowledge Graph, People Also Ask, news, video, and related-search data.

Q: Is SERP API JSON the same as raw HTML?

A: No. JSON exposes parsed fields for application use. Raw HTML preserves the returned page markup and is useful only when the structured fields do not contain the detail your workflow needs.

Q: How should I parse a missing SERP feature in Python?

A: Treat optional arrays as empty lists and optional objects as None. A missing PAA or Knowledge Graph section can be a normal result for that query, not an API failure.

Q: Can I use SERP JSON for a RAG pipeline?

A: Yes. Use the JSON response to discover and rank candidate URLs, then extract the selected pages with a Reader step. Keep the query and source URL with the Markdown context so the model can separate search observations from source content.

Q: Where can I test a SearchCans request?

A: Use the SearchCans API Playground to inspect a request and its response before integrating the same JSON shape into your Python service.