Automated Patching with Ansible and the TekStream MSP

By: Karla Broadrick | Technical Architect & Team Lead

 

TekStream is excited to roll out the use of Ansible for automated patching of customer environments managed under the Managed Services Program (MSP).

What is Ansible?

Ansible is an IT automation engine.  It can be used to automate any number of repetitive system administration tasks such as infrastructure provisioning, configuration management, task automation, and application deployment.  It can be utilized in both On-Prem and cloud environments.  Instructions are organized into roles and playbooks, allowing flexibility for tasks to be performed based on a set of input parameters.  The setup for Ansible is simple and doesn’t rely on an agent on remote hosts.  All of this leads to a great degree of flexibility in how and where Ansible is used to automate every day IT tasks.

How is TekStream utilizing Ansible?

TekStream has created an Ansible playbook that can be used to automate OS and application patching in customer’s environments.  The playbook automatically creates backups, downloads designated patches, shuts down the application, applies both Windows and Linux OS patches, applies WebLogic and other Oracle patches and reboots the host as needed.

The TekStream Ansible solution for patching is targeted at OS patching and Oracle WebCenter and WebLogic application patching, but has the flexibility to be extended into other areas.  TekStream uses this solution in our MSP customer environments.

Benefits:

The benefits to TekStream MSP customers of using Ansible for patching are many including

  • Consistency: Patches are applied the same every time in every environment. This means less mistakes are made and ensures uniformity and quality that our MSP customers can rely on.
  • Time savings: Automation saves time. No longer is there a need to manually log into each server, download and transfer patches, apply each one, and wait in between each step.  With the TekStream Ansible solution for patching, a single command kicks off the entire process.  Our MSP customers can rest assured that their downtime is minimized.
  • Scalability: This automated solution lends itself to be scaled quite easily.  Whether you have a single server environment or a large cluster, the Ansible playbook can be utilized.  The servers can be patched in parallel, without the bottleneck of required manual intervention.  This means that we are able to patch a large clustered environment with a single period of downtime.
  • Easy knowledge transfer/training: Because Ansible takes care of 99% of the work, there is very little knowledge required to do patching. Once configured for a customer environment, the solution is able to run on its own.  This makes knowledge transfer and training incredibly simple.  It also eliminates the fear of lost knowledge during personnel changes.
  • Reliability: All of the benefits listed above equates to reliability. Your organization can rest assured that patching will be done correctly, every time, and in an acceptable timeframe.
  • Cost Savings: Ansible allows us to do more in less time.  For TekStream MSP customers, this translates to cost savings and allows us to stretch your dollars further.

Interested in streamlining the patching and maintenance of your system?  Contact TekStream to learn about how we can help with Ansible and our MSP program.

Implementing Right-Click Integrations in Splunk

By: Eric Howell | Splunk Consultant

 

Splunk provides Admins the opportunity to build a huge variety of customizations, visualizations, apps, and a near-infinitude of different options to finely tune your Splunk experience and have it best suit your needs. There is a use case and a solution for almost any task you can throw at Splunk. A use case that has been brought up frequently is to implement a “Right-Click Integration” to allow information from Splunk to be passed to another tool. Originally, this seemed a daunting task that could require the implementation of custom java script or a unique python script to solve for. Ultimately, the solution was much simpler and the logic is already found in Splunk – Workflow Actions!

Workflow actions allow for Splunk to perform a variety of tasks against available web resources for information found in ingested field/value pairs. They can be very easy to set up either through the UI or by deploying a workflow_actions.conf, as best depends on the size of your Splunk environment, and they offer quite a bit of additional functionality. Utilizing this functionality, your “Right-Click” integration may require an extra click, but it will be fully functional and should be fully serviceable.

Leveraging Workflow Actions to Pass Values and Fields

As discussed above, to set up a workflow action, an admin can leverage the WebUI on one of the Search Heads in an environment or deploy the workflow_actions.conf file as appropriate. This will allow a user to expand an event and pass a value from a field to an external source. In the examples below, we will use VirusTotal as an example.

WebUI Setup

For this example, we will be posting the value of a field to VirusTotal, allowing us to search their IOC database and verify if the value we are passing is a known, bad entity.

  1. Navigate to the Settings Dropdown in the Splunk navigator (In the top right by default) and select Fields under Knowledge

2. Next, you will select “Add new” in the “Workflow actions” category

3. This next step encompasses what this workflow action will do. We begin with creating a Label for the action, which will be what users see when they move to access it. Note, you can leverage             tokens in this descriptor to input values dynamically, based on what is clicked. In the case here, it will show the specific value that is being presented to VirusTotal.

4. Then you will indicate, in a comma-separated list, the fields that this workflow action can apply to. Leaving this blank will appear in all field menus.

 

5. If needed, the next step will be to select which eventtypes this workflow_action is applicable for. For additional reading on eventtypes: Follow this Link

a. Eventtypes are categorizations of data input by the user or an admin. An example would be when a specific IP address is found in your web data (sourcetype=access_combined src_ip=127.0.0.1), and you want to apply specific categorization to data that meets that criteria so that it appears with a new field/value pair (e.g. eventtype=local_access)

6. Next you will choose if the action is available in the Event Menu, Fields Menu, or both. This selection is a dropdown. Additional reading available here: Follow this Link

a. Event Menu – Will allow for you to apply this workflow action on the whole event. When an event returned by your search is expanded, the “Event Actions” dropdown will hold any workflow action with “Event Menu” selected here.

b. Fields Menus – Similar to the above, on an expanded event, the workflow action can be accessed by clicking the dropdown under “Action” in the list of included fields to allow one to apply the workflow action directly to the value of a specific field.

7. Next is to choose the Action type – meaning whether the workflow action acts as a “link” to another web location or whether it will run another “search”

8. The next step is configuring the properties based on which of the above Action Types was chosen

a. Link Configuration – Allows you to specify a URI (also accepts tokens), decide whether to open the link in a new window, select whether you intend to utilize a POST or GET method, and which arguments to pass to the web site.

b. Search Configuration – Allows you to specify a search string (token-friendly), dictate which app to run the search in, determine a name of a view (page) for the search to open in, whether to run the search in the same window, and input a timeframe.

9. Next click Save

 

Your workflow action is ready to test. Here is an example of what you can see when looking in an event view with an expanded event.

Deploying a workflow_actions.conf

Like most other configuration files, workflow_actions.conf is easily deployed via the method of your choosing (manually, scripted, or through the use of the Deployer). Below is what will be generated by Splunk in $SPLUNK_HOME/etc/system/local when created through the web UI:

 

[VirusTotalPost]

display_location = both

label = Search VirusTotal for $@field_value$

link.method = post

link.uri = https://www.virustotal.com/en/search/

link.target = blank

link.postargs.1.key = query

link.postargs.1.value = $@field_value$

type = link

 

This same information is deployable in an app of your choosing to $SPLUNK_HOME/etc/apps/ as appropriate for environments with clustered Search Heads or through other means of configuration management.

There are a wide variety of possible options with the stanza, and I recommend reviewing Splunk’s documentation regarding the specific .conf file here:

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Workflow_actionsconf

Leveraging your workflow_actions with older versions of Splunk Enterprise Security

If you are utilizing a version of the Splunk Enterprise Security app prior to 5.3, even if your workflow actions have global permissions, the app will not import configurations from others unless they follow a strict naming convention. If you want your workflow action to be accessible from the Incident Review dashboard, for example, you will need to make sure your app follows the one of these naming conventions:

  • DA-ESS-*
  • SA-*
  • TA-*
  • Splunk_SA_*
  • Splunk_TA_*
  • Splunk_DA-ESS_*

 

Want to learn more about right-click integrations in Splunk? Contact us today!