Open topic with navigation

# Overriding Application Properties

Each Transformation Hub module (Kafka, Zookeeper, and so on) has many additional properties available, and a system administrator might be required to override the default values for some of these properties. This section covers how to override these property values.

Property values (for properties that support overrides) are set by injecting environment variables in the respective container's start-up environment. These variables are read from a user-supplied properties file, in a specific location on the Network File Server (NFS). To see the available properties for override, consult the respective module's published documentation.

Note that in most cases, this feature is not required for normal operation of Transformation Hub, and most likely will be used at the direction of technical support. Not all properties support overrides; please check with technical support before making any changes to your configuration.

- For Kafka, ZooKeeper, and Schema Registry properties, consult the appropriate Confluent documentation.
- The properties for routing processor, stream processor and web service modules are detailed below.

Note: Legacy properties suffixed with `arcsight.eventbroker` will continue to function as they did in previous versions, but as explained below, newly added properties must be suffixed with `arcsight.th`. If two properties of the same name are set with different suffixes, the property with `arcsight.th` will supersede the other one.

- Routing Processor and Stream Processor Properties
- Web Service Properties
- Configuring the Values
- Changing Value Examples

## Routing Processor and Stream Processor Properties

As explained in Configuring the Values, prefix these properties with `arcsight.th.sp.` to create an override.

Property Name | Default Value | Description
--- | --- | ---
`RETRIES` | 2147483647 | The number of retries for broker requests that return a retry-able error.
`RETRY_BACKOFF_MS` | 100 | The amount of time (milliseconds), before a request is retried. This applies if the retries parameter is configured to be greater than 0.
`RECEIVE_BUFFER_BYTES` | 65536 | The size of the TCP receive buffer to use when reading data. If the value is -1, the OS default will be used.
`MAX_PARTITION_FETCH_BYTES` | 1048576 | The maximum amount of data per-partition the server will return. Records are fetched in batches by the consumer.
`MAX_REQUEST_SIZE` | 1048576 | The maximum size of a request in bytes.
`BUFFER_MEMORY` | 33554432 | The total bytes of memory the producer can use to buffer records waiting to be sent to the server.
`BATCH_SIZE` | 16384 | The default batch size in bytes when batching multiple records sent to a partition
`LINGER_MS` | 100 | The producer will wait for up to the given delay to allow other records to be sent so that the sends can be batched together
`HEARTBEAT_INTERVAL_MS` | 1000 | The expected time (milliseconds) between heartbeats to the consumer coordinator when using Kafka's group management facilities. Heartbeats are used to ensure that the consumer's session stays active and to facilitate rebalancing when new consumers join or leave the group.
`MAX_POLL_INTERVAL_MS` | 3600000 | The maximum delay (milliseconds) between invocations of poll() when using consumer group management
`MAX_POLL_RECORDS` | 100 | The maximum number of records returned in a single call to poll().
`SESSION_TIMEOUT_MS` | 180000 | The timeout (milliseconds) used to detect client failures when using Kafka's group management facility
`REQUEST_TIMEOUT_MS` | 305000 | The configuration controls the maximum amount of time (milliseconds) the client will wait for the response of a request.
`CONNECTIONS_MAX_IDLE_MS` | 540000 | The maximum amount of time (milliseconds) before idle connections are closed.
`TH_NUM_THREADS` | 6 | The number of threads to execute stream processing.
`MAX_BLOCK_MS` | 60000 (milliseconds) | Controls how long the KafkaProducer's send(), partitionsFor(), initTransactions(), sendOffsetsToTransaction(), commitTransaction() and abortTransaction() methods will block.
`DELIVERY_TIMEOUT_MS` | 120000 (milliseconds) | An upper bound on the time to report success or failure after a call to send() returns. This limits the total time that a record will be delayed prior to sending, the time to await acknowledgement from the broker (if expected), and the time allowed for retriable send failures.

## Web Service Properties

As explained in Configuring the Values, prefix the names of these properties with `arcsight.th.web-service.` to create an override.

Property Name | Default Value | Description
--- | --- | ---
`WS_AUTH_ARCMC_CONNECTION_TIMEOUT` | 30000 | The amount of time (milliseconds) before a request to connect to ArcMC is retried due to ArcMC timeout.
`WS_AUTH_ARCMC_CONNECTION_NUM_RETRIES` | 2 | The number of times that TH retries to connect ArcMC due to ArcMC timeout.

## Configuring the Values

1. Create a file named `arcsight-env-override.properties` under the folder `<NFS_root_DIRECTORY>/transformationhub/config`.
The `<NFS_root_DIRECTORY>` path is described in this guide as the NFS root folder (usually `/opt/arcsight/nfs/volumes`). For more information, refer to the section Creating the NFS Shares.
2. Add properties to the file. To each property, add the module prefix from the table below.

Module | Prefix
--- | ---
Kafka | `arcsight.th.kafka.`
Schema Registry | `arcsight.th.schema-registry.`
ZooKeeper | `arcsight.th.zookeeper.`
Routing/C2AV/Enrichment Processor | `arcsight.th.sp.`
Web Services | `arcsight.th.web-service.`

3. Delete the pods for which properties were defined, or, alternatively, redeploy Transformation Hub.
4. To verify the changes, search the log file (after the container's status is back to Running) for matching properties.

## Changing Value Examples

To change the value of `ZOOKEEPER_MAX_CLIENT_CNXNS` to 65, in ZooKeeper, and to change the value of `SCHEMA_REGISTRY_KAFKASTORE_TIMEOUT_MS` in the Schema Registry, create a file, <`NFS Volume mount>/transformationhub/config/arcsight-env-override.properties`, and add the following lines:

    arcsight.th.zookeeper.ZOOKEEPER_MAX_CLIENT_CNXNS=65

    arcsight.th.schema-registry.SCHEMA_REGISTRY_KAFKASTORE_TIMEOUT_MS=20000

Example of verifying the change by searching the log:

`kubectl -n transformationhub1 logs th-zookeeper-0 | grep ZOOKEEPER_MAX_CLIENT_CNXNS`
`Environment override script set: ZOOKEEPER_MAX_CLIENT_CNXNS=65
ZOOKEEPER_MAX_CLIENT_CNXNS=65`