Skip to content

web-fetch

Fetch content from a URL.

Category: Read-only

Parameters

url (required)
The URL to fetch. Must start with http:// or https://.
format (optional)

Output format. Default: markdown.

Value Description
markdown Converts HTML to Markdown (scripts, styles, and iframes stripped)
html Raw HTML
text Plain text with all markup stripped
timeout (optional)
Request timeout in seconds. Range: 1–120. Default: 30.

Output

The fetched content in the requested format. The content is truncated to about 128 KB (≈32K tokens) by default and a note is appended; the cap is configurable via toolLimits.webFetchMaxBytes in knosh.json.

Security / SSRF

To prevent a caller-supplied URL (typically reaching the tool via prompt injection) from being used to reach internal network resources, web-fetch applies an SSRF egress guard to every request, configurable via webFetch in knosh.json:

  • Host allowlist. When webFetch.allowedHosts is non-empty, only hosts in the list (matched case-insensitively) may be fetched; any other host is refused. The default, an empty list, allows any host.
  • Internal-address blocking. By default (webFetch.allowInternalHosts: false), a request is refused if its host resolves to a loopback, link-local (including the 169.254.169.254 cloud-metadata address), site-local, any-local, multicast, or IPv6 unique-local (fc00::/7) address. Set webFetch.allowInternalHosts to true to opt out of this check when an agent needs to deliberately reach an internal resource, such as a local dev server or an intranet wiki; the host allowlist, if configured, still applies.
  • Redirects re-validated per hop. web-fetch disables automatic redirect following and instead follows redirects itself, re-running both checks above against the target of every hop (up to an internal cap of 5 hops). This stops an allowlisted URL from redirecting into a disallowed internal address.

Known limitation: DNS rebinding. The guard resolves a host, validates the resolved addresses, then hands the URL back to the HTTP client, which re-resolves the host at connect time. A hostile DNS resolver could in principle return a safe address to the guard's check and a different, internal address at connect time (a TOCTOU / DNS-rebinding attack). Closing this gap would require connecting to the pinned IP address with a Host header override, which is not currently implemented.

Limitations

  • Images are not supported and return an error.
  • Returned content cap: ~128 KB / 131072 bytes by default (≈32K tokens), configurable via toolLimits.webFetchMaxBytes. A separate internal 5 MB limit bounds the raw download.
  • Timeout range: 1–120 seconds.
  • See Security / SSRF above for the DNS-rebinding limitation of the egress guard.