ITSI Aggregation Policy [Adding Additional Alert Actions]
by: Brent Mckinney | Senior Splunk Consultant
By default, ITSI allows you to perform several actions around episodes created from notable events. Under the Action Rules tab in our ITSI Aggregation Policy, we can see a list of these default actions. This blog will walk through where these actions are configured, and how to add additional alert actions to this list.
For Splunk Enterprise, command line access will be required
For Splunk Cloud, a support ticket will be needed to make the following config changes.
These alert actions can be found under the SA-ITOA app on the ITSI Search Heads. We can view the default action configs by navigating to splunk/etc/apps/SA-ITOA/default and viewing notable_event_actions.conf. Here we see a stanza for each of the default actions available. For any additional actions, we simply need to add a stanza to this configuration and enable it.
As a prerequisite, for any alert action that you wish to add to your aggregation policies, the alert action must exist on the ITSI Search Head. Whether from a custom-built action, or a supporting Splunk add-on. In this example, we’ll be adding a Slack alert action to the aggregation policy, so the “Slack Notification Alert” app is installed on our search head.
First, we’ll need to get the name of the alert action by checking alert_actions.conf. For our slack example, we’ll navigate to the Slack Notification Alert app, which can be found under splunk/etc/apps/slack_alerts. In the default directory, there is a conf file named “alert_actions.conf” that contains the name of the alert action as the stanza name, and the configuration settings below. We’ll grab the stanza name to use in ITSI. For this example, the stanza name is “[slack]”.
Note that all backend changes to configs should be made under the local directory, never default, so we’ll want to make this change in splunk/etc/apps/SA-ITOA/local/notable_event_actions.conf. You may need to create the local directory and notable_event_actions.conf if it does not already exist in this app.
Once created, we will make a stanza for the alert action. I want to add a Slack alert action, so in splunk/etc/apps/SA-ITOA/local/notable_event_actions.conf I will add the following stanza and save the file.
disabled = 0
This change will require a restart of Splunk. After restart, visiting the Action Rules tab now shows the Slack action as an option.
Clicking “Configure” will show the configuration options of the alert. In this case, defining what Slack channel we want to send to, and what the messaging should be. Global settings, such as the name of the Slack Webhook URL, can be defined within the add-on itself, but overwritten of ITSI if needed.
If you need help understanding and troubleshooting the ITSI Rules Engine or if you need in-depth guidance,
let one of our experts help with your ITSI Aggregation Policy.