Custom Splunk Metadata Fields with inputs.conf

By Zubair Rauf, Team Lead Professional Services

Splunk is a powerful tool for analyzing and visualizing data, and creating custom metadata fields at index time lets you quickly enhance your analytics. Static metadata fields can be added to inputs/hosts using inputs.conf.

What is the _meta field?

The _meta field can be used to specify metadata about an input. This can also be used to specify custom metadata fields for a host. I find the _meta field particularly useful to create a custom fields to identify the source of data.

For example, you can tag events from Universal Forwards and Heavy forwarders separately. For Splunk admins, this can help them identify the origin of data. I have found this useful when migrating Splunk instances for customers or doing license analysis as well.

How is the _meta field used?

The _meta field is defined in the ‘inputs.conf’ configuration file. The _meta field can be included as part of the [default] stanza to apply to all inputs or can be added to specific inputs. Below is an example:

#inputs.conf on forwarders

## Apply to all inputs
[default]
_meta = host_type::<hf|uf|your_own_value> test_field::test_value

## Apply to a single input
[<custom_input>]
_meta = test_field1::test_value1 test_field2::test_value2

In the above example, under [default] stanza, I created a new field named host_type and can assign a value to it. Similarly you can create other fields in the format listed above.

The [<custom_input>] stanza can be any other stanza in inputs.conf

After creating the field in inputs.conf on the forwarder, you need to define the indexed fields in fields.conf on the Search Heads and Indexers (stand-alone or clustered). Below is an example of fields.conf entry:

##fields.conf on Indexers & Search Heads

[host_type]
INDEXED = true

[test_field]
INDEXED = true

[test_field1]
INDEXED = true

[test_field2]
INDEXED = true

In the screenshot below, you can look at interesting fields and you can see the fields host_type and test_field that we just created using the config files.

You can also use tstats to run fast searches using these indexed fields. In the below screenshot, you can see the host_type field assigned in the [default] stanza in inputs.conf to tag all events from the hosts with a host_type value.

The _meta field is an important part of Splunk, and it provides a powerful way to classify and organize your data within the platform. By specifying metadata about your inputs, you can more easily search, filter, and visualize your data, and you can also fine-tune your data inputs to improve the performance of your Splunk instance.

Please reach out to us using the form below to learn more about _meta and how we can help you implement better analytics in your organization. Happy Splunking!