Open topic with navigation

# ArcMC Network Models

This section describes the format for ArcMC-pushed network model files. Two files are required, named networks.csv and zones.csv. They are CSV (comma-separated value) files, and as such can be created with a spreadsheet like Excel, or with a plain text editor like Notepad or vi.

Exact format is extremely important to effective use of these files, and very minor discrepancy can cause the some networks and zones or even the entire model to be ignored.

## `networks.csv`

The `networks.csv` file defines the networks that will be used in the`zones.csv` file.

Header Line: The header (first) line of this file should be exactly this string of text (use the Copy button to copy the string to clipboard):

    #Type,Name,Parent Group URI,Customer URI

There must be no extra whitespace characters at the beginning, end, or in the middle.

Additional Lines: The remaining lines of this file also have exactly four columns, and each line defines a network:

1. The first column, Type, must be either `Global` or `Network`. Like for the header line, extra whitespace is not allowed, but differences in case are. `Global` should appear on at most one line, since there is only one global network, which is used for defining the system zones. `Network` must be used for defining all other networks, which are used for defining user zones.

2. The second column, Name, defines the name of this network. More accurately, it defines the last part of the network's URI, since the full URI of the network is created by concatenating the parent group URI from the following column with this name.

3. The third column, URI, is the URI of the parent group in the network URI hierarchy. This is the first part of the network URI.

4. The final column, Customer URI, is optional, but if it is omitted the comma after the parent group URI is still required, so that all lines have four columns. It must be blank for the `Global` network line, and may be blank for lines where the type is `Network`. If not blank, this is the value expected in the customerURI field of events – any events that do not have this value in the customerURI field will not consider this network (or the zones within it).

For example, a line like this would define the global (system) network and call it `/All Networks/Global:`

`Global,Global,/All Networks/,`

A line like this would define a non-global (user) network attached to the customer with URI `/All Customers/Customer 1` and call it `/All Networks/Network 1`:

`Network,Network 1,/All Networks/,/All Customers/Customer 1`

And a line like this would define a non-global (user) network with no specific customer attached and call it `/All Networks/Network 2:`

`Network,Network 2,/All Networks/,`

When you are creating the zones in the `zones.csv` file, described below, you need to know the exact full URI of the networks you have defined in this file, so that each zone is connected to a network that you have defined. Any zones connected to unknown networks (or more likely to networks which have been incorrectly specified) cannot be used.

The order that the non-global (user) networks (those with `Network` in the first column) appear in the `networks.csv` file is the order that they will be considered, which matters because the zones defined in one network are allowed to overlap with the zones defined in other networks. In such a case the zone from a network that is earlier in `networks.csv` will supersede the other zones.

## zones.csv

The `zones.csv` file defines the zones within the networks already defined by the networks.csv file.

Header: The header (first) line of this file should be exactly this (use the Copy button to copy the string to clipboard):

    #Name,Start Address,End Address,Parent Group URI,Network URI

There can be no extra whitespace characters at the beginning, end, or in the middle.

Additional Lines: The remaining lines of this file also have exactly five columns, and each line defines a zone:

1. The first column, Name, defines the name of this zone. More accurately it defines the last part of the zone's URI, since the full URI of the zone is created by concatenating the parent group URI from the fourth column with this name.

2. The second column, Start Address, is the starting IP address of the range for this zone. It can be IPv4 format (such as "192.168.0.0") or IPv6 (such as "fe00::"). This is inclusive, so this address is part of the range.

3. The third column, End Address, is the ending IP address of the range for this zone. It can be in IPv4 format (such as "192.168.255.255") or IPv6 (such as "fe7f:ffff:ffff:ffff:ffff:ffff:ffff:ffff"), but must match the type of the starting IP address in the previous column. This is also inclusive, so this address is part of the range.

4. The fourth column, Parent Group URI, is the URI of the parent group in the zone URI hierarchy. This is the first part of the zone URI.

5. The fifth column, Network URI, is the network URI for the network to put this zone into. This must exactly match (including letter case) the concatenated URI from a network defined in the `networks.csv` file (the third column and second column, concatenated, from one line in that file).

For example, a line like this would define a global (system) zone containing the IP addresses from 192.168.0.0 to 192.168.255.255 and call it `/All Zones/Zone 192.168` (As shown in the example in the`networks.csv` section it will be named `/All Networks/Global`):

`Zone 192.168,192.168.0.0,192.168.255.255,/All Zones/,/All Networks/Global`

A line like this would define a non-global (user) zone containing the IP addresses from 192.168.0.0 to 192.168.0.255 and call it `/All Zones/Zone 192.168.0` This assumes that the /`All Networks/Network 1` network exists, as shown in the example in the prior section – note also that this zone overlaps with the global (system) zone `/All Zones/Zone 192.168`, but that is permissible because they are in different networks:

`Zone 192.168.0,192.168.0.0,192.168.0.255,/All Zones/,/All Networks/Network 1`

And a line like this would define a non-global (user) zone containing the IP addresses from 192.168.0.0 to 192.168.1.255 and call it `/All Zones/Zone 192.168.0-1` . This assumes that the `/All Networks/Network 2` network exists, as shown in the example in the prior section – note also that this zone overlaps with the other two zones, but that is permissible because they are in different networks:

`Zone 192.168.0-1,192.168.0.0,192.168.1.255,/All Zones/,/All Networks/Network 2`

Note that the network for `/All Zones/Zone 192.168.0`, which is `/All Networks/Network 1`, is attached to the customer with `URI /All Customers/Customer 1`, so that zone will only be applied to events for that customer. In contrast, the /All Zones/Zone 192.168.0-1 zone is not attached to a specific customer, because `/All Networks/Network 2` is not.

Unlike for `networks.csv`, the order of lines in `zones.csv` does not matter.

## Putting It All Together

Here is the complete example `networks.csv` file from the examples above:

    #Type,Name,Parent Group URI,Customer URI

    Global,Global,/All Networks/,

    Network,Network 1,/All Networks/,/All Customers/Customer 1

    Network,Network 2,/All Networks/,

And here is the corresponding example `zones.csv` file:

    #Name,Start Address,End Address,Parent Group URI,Network URI

    Zone 192.168,192.168.0.0,192.168.255.255,/All Zones/,/All Networks/Global

    Zone 192.168.0,192.168.0.0,192.168.0.255,/All Zones/,/All Networks/Network 1

    Zone 192.168.0-1,192.168.0.0,192.168.1.255,/All Zones/,/All Networks/Network 2

Consider how the addresses in different events would be mapped to zones, given these examples:

deviceAddress (in) | customerURI (in) | deviceZoneURI (out, assuming this field was not already set) | Comments
--- | --- | --- | ---
192.169.0.0 |  | /All Zones/ArcSight System/Public Address Space Zones/ARIN/192.169.0.0-192.255.255.255 (ARIN) | From the AddressBasedZonePopulator component becausethis address does not appear in this network model
192.168.2.0 |  | /All Zones/Zone 192.168 | From the global (system) network because neither of the non-global (user) zones apply to this address
192.168.1.0 |  | /All Zones/Zone 192.168.0-1 | From the non-global (user) /All Networks/Network 2 network because the event did not specify a customer
192.168.0.0 |  | /All Zones/Zone 192.168.0-1 | From the non-global (user)/All Networks/Network 2network because the event did not specify a customer
192.168.0.0 | /All Customers/Customer 1 | /All Zones/Zone 192.168.0 | From the non-global (user) /All Networks/Network 1 network because the event did specify that customer
192.168.1.0 | /All Customers/Customer 1 | /All Zones/Zone 192.168 | From the global (system) network because the eventdidspecify a customer, but the network associated with that customer did not include this address

## Troubleshooting

The following tips may be useful for troubleshooting issues with these files.

- Events that have addresses that fall outside of the zones defined by the network model will not be changed, so defining a network model will not necessarily affect all events.

- Make sure you actually pushed the network model. Look for a log line like this at the approximate point in time when you think you pushed it:

    [2023-08-28 16:42:42,775][INFO ][default.com.arcsight.agent.loadable._AuthenticationHandler][invoke] Processing request from [15.116.133.207] for service method [pushNetZoneFiles]

- In the get status results for the destination in question, or in the logs for the logStatus method (verify that AgentId is what you expect if the connector has multiple destinations), look for the `AddrBasedSysZonePopRows` and`AddrBasedUsrZonePopRows` items. If both are zero and you pushed a network model, there is something wrong. Here is a heavily-abbreviated log message, which was with the example network model from above:

    [2023-08-30 15:51:37,115][INFO ][default.com.arcsight.agent.fl][logStatus] {AddrBasedSysZonePopRows=1, AddrBasedUsrZonePopRows=2, AgentId=3Ukhcc2IBABCAAvh8rfSJOw==}

Look for errors in the log at the approximate point in time that the network model was pushed, or when the connector is starting (since it has to reparse the network model each time it starts). Here are some examples:

    [2023-08-23 14:31:05,299][ERROR][default.com.arcsight.agent.da.a.a][processNetworkTokens] Blank network name

    [2023-08-23 14:31:05,299][ERROR][default.com.arcsight.agent.da.a.a][processNetworkTokens] Multiple Global networks defined – [/All Networks/Global] will be used and [/All Networks/System] will be skipped

    [2023-08-23 14:31:05,299][ERROR][default.com.arcsight.agent.da.a.a][processNetworkTokens] Network [/All Networks/Test with cust] appears multiple times

    [2023-08-23 14:31:05,300][ERROR][default.com.arcsight.agent.da.a.a][<init>] Invalid zones.csv header [Name,Start Address,End Address,Parent Group URI,Network URI]

    [2023-08-24 14:30:05,500][ERROR][default.com.arcsight.agent.da.a.a][processZoneTokens] Unknown network name [/All Networks/Local/] in zone [/All Zones/Zone 2c10::] definition

If the events are coming into the connector with zones already set (as is very often the case for events in CEF format, for a common example), the connector leaves those zones unchanged by default as stated above. In this case the zones defined in the pushed network model not being used is not a bug, but by design. If you want the connector to discard incoming zones and apply new ones where applicable, you need to set "Rezone (override)" instead of "Normal" for the Zone Population Mode in the Network group of the destination parameters. This can be done using ArcMC, or directly on the connector by running setup.