Open topic with navigation

# Incremental Synchronization

The Web Connector supports incremental synchronization. The first time the connector synchronizes with a web site, it ingests all of the content that is requested by the task configuration. Subsequently, the connector only ingests information that is new or has changed.

The connector stores the date and time when it requests a page. The next time it requests the page, it uses this information to set the `If-Modified-Since` header in the request sent to the web server. If the web server supports the `If-Modified-Since` header and does not consider the page to have been modified, it returns an HTTP 304 response code, which tells the connector that the page has not changed. Otherwise, it returns the page.

If a page is returned, the connector performs a check to determine whether the page content has changed. Only if the connector determines that the page has changed does it ingest the page. This check occurs after clipping (if you have configured clipping) so that changes to parts of the page that are clipped do not cause the page to be re-ingested.

TIP:

Many web pages have dynamic content, and can appear to change every time they are requested.

© 2017 Hewlett Packard Enterprise Development LP | Send documentation feedback to HPE Close Send documentation feedback To open the configured email client on this computer, open an email window. Otherwise, copy the information below to a web mail client, and send this email to AutonomyTPFeedback@hpe.com. We appreciate your feedback! Product: Topic Title: Help Topic: Feedback:

_HP_HTML5_bannerTitle.htm