Increasing import performance

While the import process is very powerful, it must obey Content Manager's rules for creating records.

This introduces a significant overhead, as each record has various attributes validated, such as record number, container existence, retention triggers, security, etc.

There are several ways to improve the time for loading. As each import process is a site-specific task, we cannot give metrics for this improvement.

  • Disable the indexes not required for the duration of the import.

    The table TSRECORD contains a number of indexes whose sole purpose is to improve searching times for records based on specific search criteria.

    With these indexes not in place, the time it takes to import each record reduces slightly.

    Make sure you recreate the indexes when the import is complete.

  • Generate statistics before and after an import.

    Up-to-date statistics for the database optimizer directly affect the speed with which the database server can satisfy all the queries issued to create a logical Content Manager record.

    For long-running or large imports, consider also refreshing statistics in parallel at regular intervals throughout the import, which helps prevent skewing of the statistics as the import progresses.

    See the section for your RDBMS in Content Manager Enterprise Studio Help or CM10.1_Install.pdf for more information about generating statistics.

  • Use multiple clients for parallel importing.

    You can split a large import task between several Content Manager clients running parallel, which imports more records in the same time.

    If you use parallel importing, it is advisable to import data in sequence.

    For example, for records:

    • Boxes
    • Containers
    • Files
    • Documents
  • Run import on a server.

    If your site uses an RDBMS running on a Windows-based server, perform the import from a session of Content Manager running on the same server to greatly improve import times, as the bulk of network traffic will be eliminated from the process.

  • When importing records or locations, consider enabling the Relax Business Rules for Conversion and/or Relax Business Rules for Performance options. See Importing Records and Importing Locations for additional information.