Event filtering and aggregation

Filtering

You can add filter conditions to sort the events passed to the destination according to specific criteria during SmartConnector installation and configuration. For example, you can use filters to sort out events with certain characteristics, from specific network devices, or generated by vulnerability scanners. The events that do not meet the Connector filtering criteria are not forwarded.

To remove events that are not of interest or include only events that are of interest to your organization before they are ingested, you can use Customized Events Filtering.

For more information about configuring Filtering, see Managing SmartConnector Filter Conditions.

Aggregation

The Connector can be configured to aggregate (summarize and merge) events that have the same values in a specified set of fields, either for a specified number of times or within a specified time limit.

Connector aggregation compiles events with matching values into a single event. The aggregated event contains only the values that are common to events, and the earliest start time and latest end time. This reduces the number of individual events that must be evaluated. An event that repeats every 500 ms, for example, can be represented by a single event that is generated every 10 seconds, producing a 20:1 event compression. Individual connectors can be configured to aggregate events, thus reducing event traffic to the ESM Manager and the storage requirements in the ESM database.

For example, if the connector is configured to aggregate events with a certain Source IP and Port, Destination IP and Port, and Device Action whenever the events occur 10 times in 30 seconds. If 10 events with these matching values are received by the connector within that time frame, they are grouped into a single event with an aggregated event count of 10.

If the 30-seconds time frame expires and the connector receives only two matching events, the connector creates a single aggregated event with an aggregated event count of two. If 900 matching events are generated during 30 seconds, the connector creates 90 aggregated events, each with an aggregated event count of 10.

Firewalls are a good candidate for aggregation because of the volume of events with similar data coming in from multiple devices.