web-fetch-tool

web_fetch retrieves a URL over HTTP and returns text output.

The tool uses context-budget-aware caps and stops reading large responses once the retention window is full, so a single fetch does not buffer an entire page in memory. When direct HTTP fetches hit a bot wall or JS-only shell, it can fall back to the published dendrite-scraper CLI.

Parameters

Parameter	Type	Default	Description
url	string	—	URL to fetch
maxBytes	number	adaptive	Hard upper bound for tool output; may be reduced by safety budget
format	string	"text"	Output hint: "text", "markdown", or "html"
allowPackageFallback	boolean	false	Opt in to running the published dendrite-scraper package when plain HTTP is insufficient

Adaptive budget behavior

At execution time, web_fetch resolves effectiveMaxBytes from four inputs:

1. Planner envelope (if available)
2. Policy bounds (min/max per tool)
3. User maxBytes (hard upper bound)
4. Strict fallback when no envelope is available

Rules:

• User maxBytes is always an upper bound.
• Planner envelopes can reduce output further for safety.
• Unknown usage states use conservative fallback caps.

Network safety

web_fetch is limited to public http and https targets by default.

Blocked targets include:

• localhost and .local hostnames
• loopback, link-local, and private IP ranges
• hostnames that resolve to private or link-local addresses
• credentialed URLs like https://user:pass@example.com
• non-HTTP protocols

This prevents the tool from probing local services, cloud metadata endpoints, and other private-network targets via prompt injection or misuse.

Dendrite fallback

Fallback is only attempted when allowPackageFallback: true and the request is for text/markdown output where plain HTTP is likely useless:

• fetch errors
• retryable HTTP statuses like 403, 429, or 503
• HTML shells that look like Cloudflare, CAPTCHA, or JavaScript-required pages

Command resolution is:

1. dendrite-scraper scrape <url>
2. uvx --from dendrite-scraper dendrite-scraper scrape <url>

That means tallow can use the published package. You do not need a local checkout. Install it directly or let uvx run it on demand.

Default policy values

Setting	Default
minimum per-tool bytes	4096
maximum per-tool bytes	524288
unknown-usage fallback cap	32768

Tool result details

web_fetch includes audit fields in details:

• effectiveMaxBytes
• budgetLimited
• budgetReason
• batchSize
• backend
• fallbackReason
• fallbackCommand
• source

These make it clear why truncation happened, whether a planner envelope was applied, and when dendrite handled the fetch instead of plain HTTP.

When to use

• Reading documentation or articles
• Checking API responses
• Inspecting page text from a known URL