Connectors

The primary purpose of IDOL Connectors is to retrieve data from repositories.

There are IDOL connectors for over 150 different repositories, including:

• Local and network file systems.

• Internet servers, Web sites, and social media feeds.

• E-mail servers such as Microsoft Exchange, Lotus Notes.

• Database servers such as SQL Server, Oracle, or MySQL.

• Document and Content Management Systems such as Microsoft SharePoint.

• Cloud services such as OneDrive and Google Drive.

Retrieve Information

Connectors keep your IDOL index up-to-date (synchronized) with the data in your data repositories. Synchronization is usually configured to occur on a regular schedule, for example overnight when the data repository is less busy processing other requests. By default, connectors retrieve all the files from a repository. However, you can configure them to retrieve exactly the information you need, and ignore information that you do not want to index.

You can specify where to start searching for files. For example, when you configure a File System Connector you specify a folder, and when you configure a Web Connector you specify the URL of a web page or site map.

You can retrieve or ignore specific items. With some connectors you can specify a query to retrieve the information that you want. Others can ingest or ignore items based on their path, type, size, or other properties. You can configure many connectors to retrieve or ignore data when a file, database, or item name matches a regular expression.

After a connector has retrieved files from a repository, the files are copied to a local temporary folder so that they can be processed.

Incremental Synchronization

Connectors keep a record of the files that have been retrieved, and continue to monitor the data repository. If files are added, updated, or deleted, the connector automatically extracts the new information. Processing data incrementally provides much better performance because the connector does not need to retrieve the same information over and over again.

Incremental synchronization can be implemented in different ways, depending on the features offered by the data repository. In some cases the connector can query for new, updated, or deleted items. Some repositories keep a log of changes that the connector can process. Sometimes the data repository does not provide information about changes, so the connector itself keeps a record of the items it has seen.

Document Security

In addition to extracting files from repositories, IDOL Connectors are also an important part of IDOL document security. Document security ensures that IDOL does not return documents to a user unless that user is permitted to view them.

Connectors can extract security information that specifies who is permitted to view an item in a repository, and store that information in document fields in the IDOL index. When you send queries to your IDOL Content component, it uses the information to determine whether to return a document. See Document Security.

Some repositories also maintain their own database of users and groups. Connectors can extract information about security groups, and send that information to an OmniGroupServer.

Additional Features

Keeping IDOL up-to-date with the information in your data repositories is just one task (synchronize) that IDOL Connectors can perform. You can also use connectors to perform other tasks:

• View items from a repository. A user of a front-end application performs a search and is presented with query results. After selecting one of the results the user might want to view the original file from the data repository. An IDOL Connector can be used to retrieve the file.
• Collect items from a repository. A legal discovery application might query the IDOL index to find documents that are relevant to a legal case. The application can use IDOL connectors to retrieve the original files.
• Place a legal hold on items in a repository. A legal application might do this so that items relevant to a legal case cannot be deleted.
• Delete items from a repository. Micro Focus ControlPoint and other information management applications can use connectors to delete obsolete data and files that no longer need to be retained.

For a full list of connector features, see Connector Actions.