# UseSitemapFromRobots

Some `robots.txt` files include the location of the sitemap(s) for the site. This parameter specifies whether the connector should look for sitemap URLs in the `robots.txt` file.

You can only set this parameter if you have configured a single start URL with the Url parameter.

If the connector does not find the URL of a sitemap, it crawls the site by following hyperlinks from the start URL.

Type: | Boolean
Default: | False
Required: | No
Configuration Section: | TaskName or FetchTasks or Default
Example: | `UseSitemapFromRobots=TRUE`
See Also: | Url

_FT_HTML5_bannerTitle.htm