Working with Active Lists

Active lists allow you to track traffic with IP addresses of interest. While you can manually update active lists, their real value comes when you define them in conjunction with rules specifically tailored to interact with and populate the lists dynamically. Lists that are not rule-driven are empty or contain only manual entries that have not timed out.

In the ArcSight Platform, you can create and edit both event-based and field-based active lists. Viewing active list entries, however, is not supported. In the ArcSight Platform, the active lists you create are read-optimized.

With read-optimized active lists, each component accessing the list holds a local copy of the list data. The local cache provides the best performance for rule filters and data monitors that reference the list. However, changes to a read-optimized active list require a short time to propagate to each local copy, so some events might be evaluated against stale list data.

Note: In the ArcSight Platform, the available active lists are specific to the user that is currently logged in.

To create an active list:

  1. (Conditional) If you are not using ArcSight Saas, select ESM and then select Active List.

  2. (Conditional) If you are using ArcSight Saas, select Detect and then select Active List.

  3. On the My Active Lists page, click the plus (+) icon.

  4. Provide the following information:

    Name Provide a name for the active list. Special characters are allowed.
    Capacity (x1000)

    The maximum number of active list entries to keep in memory. The default is 10,000. For most cases, 10,000 is appropriate; however, you might want to adjust this setting if the devices you are monitoring for this active list contain a lot of data.

    This represents a limit on in-memory capacity only. If you also select Partially Cached, the system retains more entries but this has an impact on performance when it is necessary to retrieve active list items from the database.

    If the maximum number of entries is reached, an existing entry is randomly selected and removed. For multi-mapped lists, removal is based on the key field and starts when the number of keys exceeds capacity.

    Capacity influences the maximum memory that the active list can consume. Memory usage is proportional to the number of entries in the list, which is usually less than the capacity. Capacity affects memory usage, but has little if any impact on performance.

    TTL Days

    TTL Hours

    TTL Minutes

    		TTL (Time To Live) means the items remain in the list for at least the amount of time you specify in days, hours, or minutes. Use 0 (zero) to cause the field to never expire. The maximum number of days is 99999.
    Count Limit

    Count Limit is used to limit the number of unnecessary updates to active list entries and improve performance.

    For example, if an On Every Event rule adds an entry to a list, but additional rules only check if an entry is in the list, not the count, there is no reason to update the count field of the entry every time.

    The Count Limit is a hard limit for the maximum count for an entry. A value of 0 (zero) indicates an unlimited count.
    Case Sensitivity

    Select whether the list will be case-sensitive or case-insensitive.

    Note: After you save the list, you cannot change this setting. If you want to revert the case sensitivity setting, define a new list instead.
    Cache Model

    The Cache Model determines how list data is accessed in a distributed cluster.

    Active lists that you create in the ESM or Detect Dashboards are read-optimized.

    In the ArcSight Platform, this field is read-only.
    Multi Mapping

    Check this box to allow multiple instances of key pairings. This enables a single key, such as an actor attribute, to map to multiple values, such as a set of roles. You can use this to return a list of entries with the same value for the key field. For example, with multi-mapping enabled, you can create an active list that could return multiple roles for an actor named Clark Kent (reporter, superhero, space traveller) or multiple names associated with a farmhouse in Kansas (Clark Kent, Superman, Kal-El).

    Note: Do not select this option if you are creating a time partitioned active list.
    Partially Cached

    Check this box to allow additional entries beyond the in-memory Capacity (x1000) maximum to be stored and retrieved from the database.

    Using partial caching increases overall capacity but can impact performance because it takes more time to retrieve list entries from the database. This setting is required by active lists that are time partitioned.

    Note: There is a limitation when in-memory resources such as active channels and data monitors are used to return values from a partially-cached list. Only those values that are in the cache are returned. Reports and query viewers are not affected by this limitation because these resources query the database directly and do not use cache.
    Time Partitioned Check this box in addition to Partially Cached to capture data over time. Without time partitioning, a partially-cached list requires constant retrievals from the database to update the entries, and old entries are removed at random. With time partitioning, the cached data is segregated into partitions based on the list’s timestamp (Date field) value. Time-partitioned list data are kept in memory, and older data are the first to age out of the list.
  5. (Conditional) If you are creating an event-based active list, in the Data section, select a field category to see the available fields for that category, and then drag the fields for which you want to collect data to the Selected Fields column. When you are done, apply your changes and save the active list.

  6. (Conditional) If you are creating a field-based active list, enter the corresponding data type, sub-type, and mark as key field as required. Refer to the following table for guidance:

    IP Address

    This field supports IPv4 or IPv6 address. If the value is an IPv6 address, the resulting address is displayed in simplified format if applicable. For example, 2001:db8:0000:0000:0000 is displayed as

    2001:db8::

    Date This Date field is used as a default Timestamp value for interval-type queries on active lists.
    Double, Integer, or Long

    Select the applicable numeric type.

    Note: Leave the Subtype column blank even if you see the selections. The numeric subtypes MIN, MAX, and SUM are not supported in active lists.
    MAC Address

    MAC address of the format consisting of six groups of two hexadecimal digits per group. Use hyphen (-) as separators. For example

    01-00-5E-90-10-FF

    Resource Reference

    		Any ArcSight Resource such as asset and so on.
    String

    This is optional for lists in general but required, along with a Date field, if your list is time partitioned.
    Key field Select one or more fields that must be unique. In most cases, you would select at least two fields to make a key-value pair. For example, in the case of a DHCP login event, when a new IP and zone combination are written to the list, this indicates that a new session has started.

    Database columns are defined after the active list is created. Column definitions cannot be added, removed, or changed once the new active list is saved.

  7. Refresh the My Active Lists grid to view the new active list.

To edit or delete an active list:

  1. Select the active list from the My Active Lists grid, and then click the pencil or trash can icon.

  2. If you selected to edit an active list, you can edit the following fields:

    • Capacity (x1000)
    • TTL Days
    • TTL Hours
    • TTL Minutes
    • Count Limit