Creating or Editing an Active List

Purpose: Active lists are defined in conjunction with rules specifically tailored to interact with and populate the lists dynamically. See Using Rules to Populate an Active List.

Lists not driven by rules are empty or contain only manually added entries that have not timed out.

Where: Navigator > Resources > Lists > Active Lists tab

Procedure:

  1. To create an active list, right-click an active list group and select New Active List.

    To edit an active list, right-click an active list and select Edit Active List.

  2. Set options as follows:

    Active List Attributes

    In this field...

    ...enter this

    Name

    Enter a name for the active list. This name identifies the active list in ArcSight list selector popups. Spaces and special characters are allowed.

    Optimize Data

    If you want to create a hash-based list, click Optimize Data to toggle it on. This option reduces the memory usage of an active list. It is useful for active lists with more than 1,000 entries or for lists that contain a large amount of information per entry. See Optimize Data with Hash-Based Active Lists.

    Capacity (x1000)

    This setting indicates the maximum number of active list entries the system is to keep in memory. The default is 10,000. For most cases, 10,000 is appropriate, however, you may want to adjust this setting if the devices you are monitoring for this active list contain a lot of data to ensure you have adequate memory cache available.

    Notes:

    • This represents a limit on in-memory capacity only. If you also select Partially cached, more entries are retained but this has an impact on performance when it is necessary to retrieve active list items from the database.

    • If the maximum number of entries is reached, an existing entry is randomly selected and removed. For multi-mapped lists, removal is based on the key field; and starts when the number of keys exceeds capacity.

    • Capacity influences the maximum memory that can be consumed by the active list. The memory usage is proportional to the number of entries in the list, which usually are less than the capacity. Capacity affects memory usage, but has little if any impact on performance.

    TTL Days, TTL Hours, TTL Minutes

    		 TTL (Time To Live) means the items remain on the list for at least the amount of time you specify in Days, Hours, or Minutes. Use 0 (zero) to cause the field to never expire. The maximum number of days is 99999.

    Count Limit

    Count Limit is used to limit the number of unnecessary updates to active list entries and improve performance.

    For example, if an On Every Event rule adds an entry to a list, but additional rules only check if an entry is in the list, not the count, there is no reason to update the count field of the entry every time.

    The Count Limit is a hard limit for the maximum count for an entry.

    A value of 0 (zero) indicates an unlimited count.

    Allow multi-mappings

    Check this box to allow multiple instances of key pairings. This enables a single key, such as an actor attribute, to map to multiple values, such as a set of roles. You can use this to return a list of entries with the same value for the key field.

    For example, with multi-mappings enabled, you can create an active list that could return multiple roles for an actor named Clark Kent (reporter, superhero, space traveller) or multiple names associated with a farmhouse in Kansas (Clark Kent, Superman, Kal-El).

    Note: Don’t use this setting if you are creating a Time partitioned active list.

    Partially cached

    When Partially cached is selected, additional entries beyond the in-memory Capacity (x1000) maximum are stored and retrieved from the database.

    Using partial caching increases overall capacity but can impact performance because it takes more time to retrieve list entries from the database.

    This setting is required by active lists that are Time partitioned.

    Note: There is a limitation when in-memory resources such as active channels and data monitors are used to return values from a partially-cached list. Only those values that are in the cache are returned. Reports and query viewers are not affected by this limitation because these resources query the database directly and do not use cache.

    Time partitioned

    A partially-cached, time-partitioned active list enables you to capture data over time. Wtihout time partitioning, a partially-cached list requires constant retrievals from the database to update the entries, and flushing out old entries are done at random. With time partitioning, the cached data is segregated into partitions based on the list’s timestamp (Date field) value. Time-partitioned list data are kept in memory, and older data are the first to age out of the list.

    This option requires that:

    • The list must not be multi-mapped.

    • Partially cached must be enabled.

    • The list must be fields-based (not event-based). Fields must include at least a date and a string field that are set as key fields. Without a date key field, the time partitioned setting is ignored.

    Case Sensitivity

    You can optionally configure the list to be case-sensitive or ‑insensitive. Furthermore for case-insensitive lists, you can specify case-insensitivity for keys only, or for both keys and values. The feature enables you to store and look up values in lists regardless of case.

    Select one:

    • Case-Sensitive (the default)

    • Key Case-Insensitive

    • Key & Value Case-Insensitive

    Important: After you save the list, you cannot change this setting. If you want to revert the case sensitivity setting, define a new list instead.

    Cautions on case-insensitive active lists:

    • If your list is case insensitive, do not use the Optimize Data option.

    • Lookups on case-insensitive lists will slow down query and active channel performance. Make sure your queries and variables (used by channels) get values from case-sensitive lists.

    Cache Model

    The Cache Model determines how list data is accessed in a distributed ESM cluster.

    • When Read Optimized is selected, a local copy of the list data is held by each component accessing the lists. The local cache provides the best performance for rule filters and data monitors that reference the list. However, changes to a Read Optimized list require a short time to propagate to each local copy, so some events might be evaluated against stale list data.
    • When Write Synchronized is selected, a single cache is shared by all components, so any change to a list is simultaneously visible to all members of the cluster. However, accessing the list is slower. When using the Write Synchronized option, it is important that rule filters are structured in such a way that it minimizes access to the list. Filter clauses are usually evaluated in the order they appear in the user interface, so checking event field values such as Device Event Class ID before an inActiveList clause can reduce the overhead involved in accessing the active list.

      Note: With the Write Synchronized cache model, changing the list capacity does not take effect until the cluster is restarted.

    The Cache Model setting does not apply to lists deployed to a stand-alone ESM system.

    Common and Assign fields

    Entering data in the Common and Assign sections is optional, depending on how your environment is configured. For information about the Common and Assign attributes sections, as well as the read-only attribute fields in Parent Groups and Creation Information, see Common Resource Attribute Fields.

    Data: Event‑based, Fields‑based

    In the Data panel, choose Event-based or Fields-based lists. Your entries here determine what kinds of values your list is populated with.

    Caution: After you have selected your data fields and saved the active list, you cannot add, remove, or change existing data fields.

    • The Event-based option is convenient for choosing event attributes as found in existing events. When checking or adding to an event-based list, you only need to supply an event. This option is not supported in time-partitioned lists.

    • The Field-based option offers detailed event and attribute selection controls that involve mapping fields to field attributes. Use this setting for time-partitioned lists.

      Field-based lists that use "Key Fields" are known as active lists with values. (For more information, see Active Lists with Values.)

  3. If list data is event-based:

    1. Click Select Fields.

    2. On the Field Selector panel, select one or more event fields for your list data collection then click OK. Then click Apply or OK on the Active List Editor panel to save your event-based list.

  4. If list data is field-based:

    1. Under the Name column, replace <Enter Name> with a descriptive name for the field. For a list of restricted characters, see Field Naming Restrictions.

    2. Select the data type and corresponding subtype as applicable:

      Active List Column Types and Subtypes

      Type

      Subtype

      Date

      This field is required for time-partitioned active lists. Additionally, you must set this as a key field. If the time-partitioned list has no date or time-based field, time partitioning does not occur.

      This Date field is used as a default Timestamp value for interval-type queries on active lists.

      IP Address

      This field supports IPv4 or IPv6 address. If the value is an IPv6 address, the resulting address will be simplified if applicable. For example, 2001:db8:0000:0000:0000 will be displayed as

      2001:db8::

      Double, Integer, or Long

      Optionally select one of the numeric subtypes to accumulate values when the field is updated, for example, by a rule. If you do not select a cumulative numeric subtype, the entries are replaced when the list is updated.

      • SUM adds the existing value and the inserted value. For example, if type is Double, subtype is SUM, and the current value is 100.00, inserting a value of 50.0 results in a new value of 150.00.
      • MAX takes the greater of the existing and inserted values. For example, if type is Double, subtype is MAX, and the current value is 100.00, inserting a value of 50.0 does not change the current value because 100.00 is already a maximum of itself and 50.0.
      • MIN takes the lesser of the existing and inserted values. For example, if type is Double, subtype is MIN, and the current value is 100.00, inserting a value of 50.0 results in a value of 50.0.

      Notes:

      • The cumulative values feature is only available in fields-based active lists.

      • Do not use cumulative numeric fields as key fields.

      • If you are manually editing list entries for the cumulative numeric subtypes, the value you enter is the final value. This means accumulation of values does not occur with manual entry edits.

      • These cumulative numeric subtypes are not supported in multi-mapped active lists because new entry values for the same key add rather than modify the entries.

      • Trends cannot act on lists with cumulative numeric fields.

      • Rules and Threat Detector can act on lists that use cumulative numeric fields.

      MAC Address

      MAC address of the format consisting of six groups of two hexadecimal digits per group. Use hyphen (-) as separators. For example

      01-00-5E-90-10-FF

      Resource Reference

      Any ArcSight Resource such as asset, report, actor, and so on.

      String

      This is optional for lists in general but required, along with a Date field, if your list is time partitioned.

    3. Optionally check Key Fields to enable a per-field Key option, and then select one or more data fields that must be unique.

      Important: Key fields must have values because key fields are used to uniquely identify a record.

      For example, the ArcSight-provided active list ArcSight Foundation/Configuration Monitoring/Assets with Recent Configuration Modifications uses fields-based data, and keys on unique values for asset address, zone, and name.

      Field-based lists that use Key Fields are known as active lists with values. (For more information, see Active Lists with Values.)

      Note: For key fields, here are best practices:

      • For a time-partitioned active list, your key fields must be a Date field and a string field.

      • Do not make cumulative numeric fields as key fields.

      Database columns are defined after the list is created. After the new list is saved, you cannot add, remove, or change columns to the list.

  5. Optional: To add information in the Notes tab, refer to Using Notes.

  6. Click Apply to save and continue editing or OK to save and close.

You can use the Add Entries button in the Active List Editor to manually insert values to the current active list. See Viewing and Editing Active List Entries.