Splunk Timestamp Extraction – Where and How to Find Time!
By Tyler Phillips, Splunk Consultant
Splunk timestamp extraction can be tricky until you work with it a couple times. Follow along with me through a couple different examples and you should be able to extract a timestamp out of any data.
So, what do we use to extract a time stamp? There are a few settings in Splunk’s “Magic 8” that go into props.conf of the data you are trying to ingest. There are three settings were going to work with today.
TIME_PREFIX
TIME_PREFIX is the regex to tell Splunk where to start looking for the timestamp. Splunk will then attempt to make a timestamp out of the characters that follow the prefix. If the prefix cannot be found then timestamp extraction will not occur.
MAX_TIMESTAMP_LOOKAHEAD
This is the count of characters you want to look at in the timestamp. This count begins right after the end of the found TIME_PREFIX characters and counts from there. This does not count from the start of the event.
TIME_FORMAT
Time variables used to parse out our timestamp and tell Splunk what each item is.
So, for testing we are going to use my local Splunk install and the add data wizard/upload data to test our configs and check the data.
My first example is the splunkd_ui_accesslogs in my internal index. This one is pretty simple as Splunk actually already picks up the timestamp for us! That’s it right? Well, we could stop there and let Splunk do the work or we could tell Splunk exactly where to look and what to look at so it doesn’t have to waste resources trying to do this itself.
As you can see the correct timestamp is highlighted and yet we don’t have any settings set in advanced. Let’s change that.
First were going to take this event into a regex tester (I use regex101) to create a regex for before the timestamp. Doing this I was able to use “\s[“ as my TIME_PREFIX.
TIME_PREFIX = \s[
Next, I counted the number of characters in the timestamp and came up with 30 characters including the space. Yes, the space does count.
MAX_TIMESTAMP_LOOKAHEAD = 30
Now for the fun part. We are going to use the Splunk Time Variables doc to create a TIME_FORMAT.
Using this page, we want to find the correct variable for each element of the time and write it out in the same format with those variables. Here is what I came up with for this timestamp.
TIME_FORMAT = %d/%b/%Y:%H:%M:%S.%3N %z
The %z at the end represents time zone in the data.
Using these settings, we can see that our test worked and we are collecting the correct timestamp for Splunk so it can save on workload.
Next, I am going to use a splunkd.log event.
11-07-2023 12:09:09.683 -0500 INFO TailReader [1039908 tailreader0] – Batch input finished reading file=’/Applications/Splunk/var/spool/splunk/tracker.log’
This event has the timestamp at the beginning of the event so we are going to use the TIME_PREFIX of “^”. This tells Splunk that the timestamp is at the very beginning of the event. The methods for creating the MAX_TIMESTAMP_LOOKAHEAD and TIME_FORMAT stay the same. Using this event, I came up with these settings.
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 29
TIME_PREFIX = %m-%d-%Y %H:%M:%S.%3N %z
Next, I did want to show some syslog data and working with its timestamps. Nothing different here but syslog is a very common type of data so I thought it would be good to include. Here I will be using a couple of F5 syslog events. My settings are:
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 15
TIME_FORMAT = %b %d %H:%M:%S
If you are looking to work well with timestamps it may take a couple tries and some practice but it will be easy to pick up and make things smoother for you when bringing in data. I am happy to answer any questions you have, just contact me using the form below. Happy Splunking!