INNER CODE UNIT · Python

fetch_protected_page

tiliondev/fortress · mcp/server.py:236

async def fetch_protected_page(url: str) -> dict:
    """Fetch a web page that normal HTTP clients / headless browsers CAN'T get —
    pages behind Cloudflare, DataDome, PerimeterX, Akamai, or that return 403 /
    'Access Denied' / 'Are you a robot' / CAPTCHA challenges.

    Use this the moment a requests/httpx/fetch call is blocked, rate-limited, or
    returns a bot-detection interstitial instead of the real content. It drives a
    real recompiled-Chromium stealth browser (Fortress) that passes bot checks,
    waits out the JS challenge, and returns the resolved HTML + text.

    Returns {url, status ('ok'|'blocked'), title, text, blocked}. (Raw HTML is
    dropped from the response to save context — use extract_page for structured
    content.)
    """
    await _check_url(url)
    t = await _t()
    r = await t.fetch(url)
    r.pop("html", None)                       # agents want text, not a raw-HTML dump

View source record →