ConnectorLib Java 25.1
ConnectorLib Java SDK Programming Guide

# Incremental Synchronize Example

This section describes the connector in operation.

## Cycle 1

To begin with, the datastore table is empty. The “data” directory (the repository from which the sample connector retrieves data) contains three files:

           File1.txt
           File2.txt
           File3.txt

When the `synchronize` action runs, the connector loops over all the files. For each file, it adds a record to the table with a pending action of “Add”. When this is finished the datastore table looks like this:

Reference | LastModified | PendingAction | Seen
--- | --- | --- | ---
File1.txt | 2013-09-05 12:00 | Add | SEEN
File2.txt | 2013-09-05 12:05 | Add | SEEN
File3.txt | 2013-09-05 12:10 | Add | SEEN

The connector then updates all UNSEEN records so that their PendingAction is "Remove", but in this case no records are UNSEEN.

Next, the connector performs the actions in the table, sending an ingest add for each file. `File1.txt` and `File2.txt` are ingested successfully and the ingest result handler updates the PendingAction to “None” for those records. Normally `File3.txt` would be ingested successfully, but in this case it is not. This could be due to a network error, or the connector being stopped before the cycle is complete. As the result handler is not called (or is called but with its `success` parameter set to `false`), the PendingAction is not changed.

After the end of the first cycle the datastore looks like this:

Reference | LastModified | PendingAction | Seen
--- | --- | --- | ---
File1.txt | 2013-09-05 12:00 | None | SEEN
File2.txt | 2013-09-05 12:05 | None | SEEN
File3.txt | 2013-09-05 12:10 | Add | SEEN

## Cycle 2

At the next synchronize cycle the data directory has been modified. `File1.txt` has been deleted, and `File2.txt` has been modified. `File3.txt` has not changed.

When the `synchronize` action runs, the connector marks all records as "UNSEEN":

Reference | LastModified | PendingAction | Seen
--- | --- | --- | ---
File1.txt | 2013-09-05 12:00 | None | UNSEEN
File2.txt | 2013-09-05 12:05 | None | UNSEEN
File3.txt | 2013-09-05 12:10 | Add | UNSEEN

The connector then loops over all files in the repository, updating the PendingAction column. `File2.txt` has a different modified date so its PendingAction is set to "Replace". `File3.txt` is unchanged so the record is not modified. After this the datastore table looks like this:

Reference | LastModified | PendingAction | Seen
--- | --- | --- | ---
File1.txt | 2013-09-05 12:00 | None | UNSEEN
File2.txt | 2013-09-05 12:30 | Replace | SEEN
File3.txt | 2013-09-05 12:10 | Add | SEEN

Next, the connector sets the PendingAction for all "UNSEEN" records to "Remove":

Reference | LastModified | PendingAction | Seen
--- | --- | --- | ---
File1.txt | 2013-09-05 12:00 | Remove | UNSEEN
File2.txt | 2013-09-05 12:30 | Replace | SEEN
File3.txt | 2013-09-05 12:10 | Add | SEEN

Finally, the connector performs each of the pending actions. This time the connector is allowed to finish, there are no network errors and all of the ingest commands are successful. The ingest result handler removes the record for `File1.txt`, and sets the PendingAction of the other records to "None":

Reference | LastModified | PendingAction | Seen
--- | --- | --- | ---
File2.txt | 2013-09-05 12:30 | None | SEEN
File3.txt | 2013-09-05 12:10 | None | SEEN

Copyright 2024 Open Text
Version 25.1 | Last updated January 2025

Send documentation feedback

Close

Send documentation feedback

To send feedback using your default email client, open an email window.

Otherwise, send your feedback and the information below to MFI-swpdl.idoldocsfeedback@opentext.com.

We appreciate your feedback!

Product: Knowledge Discovery ConnectorLib Java 25.1

Topic Title: Incremental Synchronize Example