Troubleshooting Your Splunk Environment Utilizing Btool
By: Chris Winarski | Splunk Consultant
Btool is a utility created and provided within the Splunk Enterprise download and when it comes to troubleshooting your .conf files, Btool is your friend. From a technical standpoint, Btool shows you the “merged” .conf files that are written to disc and what the current .conf files contain at the time of execution, HOWEVER, this may not show you what Splunk is actually using at that specific time, because Splunk is running off of the settings that are written in memory and for your changes to a .conf file to be read from disc to memory requires a restart of that specific Splunk instance or force Splunk to reload of the .conf files. This blog is focused primarily on a Linux environment, but if you would like more information on how to go about this in a Windows environment feel free to inquire below! These are some use cases for your troubleshooting using Btool.
Btool checks disk NOT what Splunk has in Memory
Let’s say you just changed an inputs.conf file on a forwarder – Adding a sourcetype to the incoming data:
The next step would be to change directory to $SPLUNK_HOME/bin directory
($SPLUNK_HOME = where you installed splunk, best practice is /opt/splunk)
Now once in the bin directory, you will be able to use the command:
./splunk Btool inputs list
This will output every inputs.conf file that is currently saved to that machine taking the current precedence and what their attributes are. This is what will be merged when Splunk restarts and in which is written to memory, which is why the current instance running needs to be restarted to write our “sourcetype” change above to the memory so it can utilize that attribute. If we don’t restart the instance, Splunk will have no idea that we edited a .conf file and will not use our added attribute.
The above command shows us that our change was saved to disc, but in order for Splunk to utilize this attribute, we still have to restart the instance.
./splunk restart
Once restarted, all Btool merge files are in memory and describe how Splunk is currently acting at that given time.
Btool conf file creating a file with returned results
The above example will just simply print out the results to the console, where the code below will run the command and then create a file located in your “tmp” folder of all the returned text involving the inputs.conf files in your splunk instance.
./splunk btool inputs list > /tmp/btool_inputs.txt
Where do these conf files come from?
When running the normal Btool command above, we are returning ALL the settings in all inputs.conf files for the entire instance, however, we can’t tell which inputs.conf file each setting is defined in. This can be done by adding a –debug parameter.
./splunk btool inputs list –debug
Organizing the Btool legibility
When printing out the long list of conf files, they seem to be all smashed together, using the ‘sed’ command we are able to pretty it up a bit using some simple regex.
./splunk btool inputs list | sed ‘s/^\([^\[]\)/ \1/’
There are many other useful ways to utilize Btool such as incorporating scripts, etc. If you would like more information and would like to know more about how to utilize Btool in your environment, contact us today!