Skip to main content

New Crawling options in DubBot

Product Updates Icon, two arrows forming a circle

DubBot allows users to crawl websites via a specific domain and subsections off of a domain. In addition to these options, DubBot will now allow users to explicitly specify lists of URLs for crawling. This list may be provided via a URL to a sitemap, a CSV upload, or manual entry through the DubBot interface.

These new options for crawling content will allow users to more selectively curate content for auditing. DubBot users can use these options to further help prioritize content fixes.

DubBot users can find more information on how to configure these new options in the Help Center or by reaching out to our support team.

penny

This entry has 0 replies

Comments open

Leave a reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>