Add Live Web Retrieval to RAG With an Explicit Freshness Router

A live-web RAG workflow needs a freshness decision before it needs another search call. Keep the age and availability of the evidence, the task’s freshness requirement and the action taken. Reuse a source only when its stored revision remains suitable for the question.

Route reuse, refresh and removal separately

from datetime import datetime, timezone

def freshness_action(last_checked, now, ttl_seconds, availability='available'):
    if availability == 'removed':
        return 'tombstone'
    if (now - last_checked).total_seconds() > ttl_seconds:
        return 'refresh'
    return 'reuse'

now = datetime(2026, 10, 10, 12, tzinfo=timezone.utc)
checked = datetime(2026, 10, 10, 11, tzinfo=timezone.utc)
assert freshness_action(checked, now, 7200) == "reuse"
assert freshness_action(checked, now, 1800) == "refresh"
assert freshness_action(checked, now, 7200, "removed") == "tombstone"

The timestamps and TTL values are synthetic test choices, not a recommended freshness policy for every topic. A removed source follows a deletion route even if its previous timestamp is recent.

Refresh through the documented Search and Reader contracts

Use POST /api/v1/search with t and s for discovery, then select URLs from data.organic[].link. Use POST /api/v1/url with t: url and s containing a selected URL for reading. Keep the business status and extraction result separate; do not use a GET request with another provider’s q/engine envelope.

Record source URL, revision hash and access time with the accepted passage. A fresh extraction can still be irrelevant or unsupported. Check the answer against the passage before generation and allow an insufficient-evidence response.

Test the router, then evaluate the answers

The local router cases verify state transitions. They do not call a live language model or establish answer accuracy. The architecture guide covers the whole evidence pipeline.

• Cover fresh cache, expired cache, removed source and failed refresh.
• Keep stale fallback explicitly labelled and only use it where the application permits it.
• Compare answers against a frozen question/source set when evaluating quality.
• Do not infer hallucination reduction from the fact that a web request completed.

Reference documentation: www.searchcans.com — /google-search-api; www.searchcans.com — /reader-api.

Related implementation and validation guides

• Real-Time RAG Web Search: A Practical Evidence Architecture