Chaining Search Operators

Construct a complex query statement by chaining together multiple search operators into a single query instead of implementing separate queries. This powerful capability lets you perform robust, real-world searches while providing the flexibility to customize searches for specific scenarios. You can save these searches to reuse them in future updates.

Operator chaining is a process by which the search takes a set of results from one operation and uses these results as input for the next operation. Chaining a series of operations equips you with the options needed to "slice and dice" data to extract and analyze it on a highly granular level. Operator chaining works with all the pipeline operators (rename, eval, where/filter, wheresql, top, bottom/rare and chart/stats). The number of search operators supported in a particular query might vary, based on your database configuration and load.

During operator chaining, fieldsets become more restricted as more operators are added to the query, especially with eval and aggregation operators. For example:

severity!=null | top severity | stats avg (Count_1) by severity

For information about operator chaining workflows, see Use Cases for Search Operators.

Syntax Recommendations

Use the following syntax recommendations to ensure operator chained searches succeed.

• To use the fields from a lookup list table with the search operators, make a join with one of the lookup fields using the "in list" operator. You also should add the lookup list fields to the current fieldset. For example:
  
  Add a lookup list with name as Customer then add its field, which will be used with search operator (e.g. Customer_Vendor) to the current fieldset.
  

  Source Address in list Customer_Address | wheresql Customer_Vendor = 'Microsoft'

• Alias/New field name cannot be an existing field name or a synonym of an existing field name. Also, an alias field name cannot be an existing group name or reserved word.
  

  In this example, "destination hostname" and its synonyms "dhost" and "destination name" cannot be used as aliases.

• Alias/New field name should not have spaces (like test 1), otherwise it will cause conflicts. These are examples for acceptable alias/new field names:

  name is not null| eval test1 = concat(name, "_test") | eval test_2 = upper(test1)

  name is not null | where name not equals ARCSIGHT | chart count (distinct name) as Dcount by name

• For all pipeline operators, special characters used in aliases have syntax restrictions. This includes those described in the eval, chart and stats, rename, top and bottom, and where operators.

• The following is an example of how to use a generated field with eval in another operator:

  | eval test = upper ( name ) | where test != "ARCSIGHT"

• Count_<number> cannot be used as an alias for a field name.

• The wheresql operator is case sensitive. Just like all other operators, the wheresql name must be stated in all lower case letters.

• Do not create new field name with spaces if these new fields will be used later with the wheresql operator. The where condition of wheresql operator will not recognize new field with spaces that were created by previous operators. In addition to wheresql, this is also applicable for the eval and chart and stats operators. Here are two invalid examples:

  | rename name as new name | wheresql new name = 'TCP'

  destination port is not null | eval convert name = upper ( Name ) | wheresql convert name = 'MSTYPE'

• You can use the where operator to filter dynamic fields, for example:

  | top 5 Name | where Count_1 > 1000

• More filters can be added to a search through the Fields Summary feature. Click on Fields Summary and select a field and a value for that field. The new filter will be appended at the end of the query in use as a |where clause.

• Multiple aggregate functions can drastically modify drastically the fieldset that is available for the next pipe operator. for example:

  name is not null | char count (Name) by Device Vendor span 1h | chart count (Name) by Name

  The second chart pipe cannot access to span operations because the NET, DRT, dBRT are not available for this chaining level.

  Same scenario applies to the top operator:

  name is not null | top Name | char count(count_Name_1) by Name span 1h