Scan and crawl

Inclusa scans a single URL or crawls an entire site, following same origin links, headless or visible. You decide the bounds, and the crawl stays inside them.

Bounded and safe

Every crawl carries a page cap, a depth cap, and a per page delay, with an optional sitemap.xml seed to reach pages that nothing links to. The frontier is deduplicated, the same origin cap is pinned to the seeds, and scope include and exclude rules let you keep the crawl on the parts of the site you care about.

Loop and poison resistant

A deduplicated frontier means the crawl never chases its own tail, and sitemap parsing is bounded by document and page caps, a host check, and body size and timeout limits. A malicious or runaway sitemap cannot widen the scan beyond what you asked for.

Real pages

Checks run against the rendered DOM in an actual browser, so single page apps and client routed pages are checked the way a user sees them, not the way the initial HTML happens to look.