How to Build a Real-Time AI News Monitor with n8n, SERP API, and LLMs

In the age of information overload, manually refreshing Google News or Twitter to track your industry is a productivity killer. While RSS feeds were the old standard, modern dynamic websites have made them obsolete.

Today, we are going to build a self-updating AI News Agent using n8n.

Unlike basic scrapers that break whenever a website changes its <div> structure, this workflow is resilient. It uses SearchCans SERP API to find fresh URLs and the Reader API to convert complex web pages into LLM-friendly Markdown.

Why n8n + SearchCans?

Running a headless browser (like Puppeteer) inside your n8n instance is resource-heavy and prone to crashing. By offloading the “browsing” to SearchCans, you keep your automations lightweight and fast.

Discovery (SERP API)

We use this to find new content indexed by Google/Bing in the last 24 hours.

Extraction (Reader API)

We use this to turn messy HTML/JS into clean Markdown. This is critical for AI. LLMs process Markdown 30-50% more efficiently than raw HTML, saving you money on OpenAI tokens.

Intelligence (LLM)

We feed the clean text to GPT-4o or Claude 3.5 for summarization.

The Architecture

The logic for our news monitor is linear but powerful:

1. Cron Trigger: Runs every morning at 8:00 AM.
2. Search: Queries Google for specific keywords (e.g., “SaaS Pricing Models”) filtered by date.
3. **Read:**Iterates through the top 5 URLs and fetches full content via Reader API.
4. Synthesize: AI summarizes the articles into a briefing.
5. Deliver: Sends a digest to Slack or Email.

Step 1: Setting up the Search (SERP API)

First, we need to find the news. In n8n, add an HTTP Request node. We will use the SearchCans SERP API to fetch real-time results.

Node Configuration:

• Method: GET
• URL: https://www.searchcans.com/api/search
• Query Parameters:
  • query: site:techcrunch.com OR site:verge.com "AI agents" (tailor this to your niche)
  • engine: google
  • time_frame: d (past 24 hours)

// Example JSON Output from SearchCans
{
  "keyword": "AI agents",
  "result": {
    "data": [
      {
        "title": "The Rise of Autonomous AI Agents",
        "url": "https://techcrunch.com/2026/01/07/ai-agents-rise/",
        "snippet": "New frameworks are allowing agents to..."
      }
      // ... more results
    ]
  }
}

Pro Tip: With our Ultra Plan, this search costs effectively nothing ($0.56 per 1,000 searches). You can run this every hour without breaking the bank.

Step 2: Fetching Full Content (Reader API)

This is where most n8n workflows fail. Passing a raw URL to an AI node usually results in hallucination or “I cannot access this link.” Passing raw HTML results in token overflow.

We solve this with the SearchCans Reader API. It handles the headless browsing, cookie consents, and JavaScript rendering, returning clean Markdown.

Add a Loop Over Items node in n8n to iterate through the SERP results, then add another HTTP Request node:

Node Configuration:

• Method: GET
• URL: https://www.searchcans.com/api/url
• Query Parameters:
  • url: {{ $json.url }} (from the previous step)
  • use_browser: true (ensures dynamic content loads)

// The Reader API transforms the chaotic web into structure:
# The Rise of Autonomous AI Agents

> By John Doe | Jan 7, 2026

**Source:** https://techcrunch.com/...

The landscape of artificial intelligence is shifting...
[Rest of the article in clean Markdown]

Step 3: The AI Analyst (RAG Pipeline)

Now that you have high-quality, clean text, the rest is easy. Add an OpenAI or Anthropic node to your n8n workflow.

System Prompt:

You are an expert news analyst. You will be provided with the content of an article in Markdown format.
1. Extract the core news hook.
2. Analyze the impact on the [Your Industry] sector.
3. Output a concise 3-bullet summary.

Map the content field from the Reader API output to the user message in the AI node.

Optimizing for Cost and Scale

If you are building this for a production environment (e.g., monitoring 1,000+ keywords for a PR agency), efficiency matters.

1. Filter Aggressively: Use the snippet from the SERP API (Step 1) to filter out irrelevant URLs before calling the Reader API.
2. Use Markdown: As mentioned, Markdown uses fewer tokens than HTML. Over a month of running this bot, the Reader API effectively pays for itself by reducing your OpenAI bill.
3. Choose the Right Plan: If you are scaling up, our Standard Plan starts at $18, but high-volume users should look at the Ultra tier to drive per-request costs down to $0.00056.

Conclusion

Building a news monitor doesn’t require complex Python scripts or expensive subscriptions to media monitoring tools. With n8n orchestrating the flow and SearchCans handling the “dirty work” of web searching and scraping, you can build a custom intelligence engine in under an hour.

Ready to build your agent?

---

Resources

Related Topics:

• Building Advanced RAG with Real-Time Data - Learn how to vectorise this data.
• SERP API vs Web Scraping Comparison - When to switch from n8n to code.

Get Started:

• Free Trial - Get 100 free credits
• API Documentation - Technical reference
• Pricing - Transparent costs

---

SearchCans provides real-time data for AI agents. Start building now →