Use Storage Groups to Organize and Retain Data

You can divide data into storage groups, which allows you to partition the incoming events data and provide different retention periods, based on the query filter. Because you can set data retention policies per storage group, you can retain certain high volume events for a short time period and other important events for longer time period. Higher volumes of event data, require more storage space. The storage utilization column displays the amount of storage utilized.

The query filter enables you to associate a storage group with specific compliance requirements, business needs, or search activities. Your specified query filters direct events to the correct storage group. For example, one group might have a filter for categoryDeviceGroup =/ Firewall and another for severity >= 7. If an event does not match any of the active filters, the event gets sent to the Default Storage Group. You cannot change the name, query, or rank of this built-in group.

By default, the maximum value for retaining events in the Default Storage Group is 12 months. However, the license for your deployed product might require a lower maximum value, such as 30 days. For more information about how deployed products affect data retention policies in a non-SaaS environment, see "Understanding License Keys" in the ArcSight Platform Administrator's Guide.

The Apply Changes to System option at the top of the Storage Groups page indicates that one or more groups have been modified but the changes need to be applied.

Create a Storage Group

You can have up to 10 storage groups, including the provided Default Storage Group.

  1. Select Configuration > Storage.
  2. Click the add icon +.
  3. Enter a name for the storage group.
    4. CAUTION: You cannot change the name after you create the group.
    The name cannot include special characters other than a hyphen (-).

  4. Enter a query with which to filter the incoming events into this storage group.

    For example: categoryDeviceGroup='/Firewall' or categoryDeviceGroup='/IDS'.

    The query can include parentheses, quotes, and single quotes.

  5. For the storage group’s status, indicate whether to activate the group.
  6. (Optional) For Delete Data Older than, enter the age of data, in months, that you want to purge from the storage group in the database.
  7. Click Save.
  8. Apply your changes.

Direct Events to the Correct Storage Group

For efficient data retrieval, the system matches each incoming event with the query filter for a single, active storage group. However, an event could be associated with the rules of more than one group. When an event matches with multiple storage groups, the system assigns the event to the highest ranked group.

For example, if Event_29 matches the query filter for the storage groups ranked 3, 5, and 6, then the system assigns the event to the group that is ranked 3. If an event does not match any of the active filters, the system sends the event to the Default Storage Group.

You can change the ranking of storage groups to ensure that the system places events in the best location.

  1. Select Configuration > Storage.

  2. From the Storage Information table, drag each storage group up or down to the preferred priority position.

    The system always places the Default Storage Group in the lowest ranked position.