Azure data ingest changes with Splunk Enterprise/Cloud 10.2
By Kyle Moreau, Senior Splunk Consultant
Recent updates from Microsoft change the way Azure/Entra data is ingested into Splunk. This change replaces the Splunk Add on for Microsoft Azure with the following apps:
- Splunk Add on for Microsoft Cloud Services
- Splunk Add on for Microsoft Office 365
- Splunk Add on for Microsoft Security
Splunk has created a reference guide that describes where the data sources have migrated to: https://splunk.github.io/splunk-add-on-for-microsoft-cloud-services/Migrationguide/
You will notice that some of these sources have shifted to Event Hub logging before being ingested into Splunk, this is mainly due to throttling limits.
Azure lists out a pricing plan and general guidelines on determining which pricing option is best for you: https://azure.microsoft.com/en-us/pricing/details/event-hubs/
To get a good idea of what your data ingest from these sources looks like today, you can perform metrics searches to determine thruput. The search below give you an hourly representation of how much data is coming into Splunk from Azure Audit logs by the hour. Note that the search displays the average and the max MB/s each hour and displays them in a timechart. I recommend running this search against different days/weeks to get an accurate data sample.
index=_internal group=per_sourcetype_thruput series=azure:aad:audit | eval MBps = kbps / 1024 | timechart span=1h avg(MBps) as avg_MBps max(MBps) as max_MBps by series If your thruput is low enough to stay in the Basic plan, then you will need to determine the number of events you ingest each month. You can do this with the search below. Make sure to adjust the search time to be at least 1 month.
| tstats count values(sourcetype) as sourcetype where index=azure sourcetype=azure:aad:audit by index span=1mon _time
When you determine your available thruput per sourcetype and event count per month, you can go to Microsoft’s pricing calculator to determine the right Tier for you: https://azure.microsoft.com/en-us/pricing/calculator/?service=event-hubs
Once you’ve determined your Event Hub subscription, proceed to setting up Event Hub logging for your data sources. In this example we used audit logs. In Azure, navigate to the Microsoft Entra ID tab, expand Monitoring, select Audit logs, add diagnostic setting, check the stream to an event hub check box and fill in the required settings. In this section you can add all of the log categories you need sent to an event hub. Select save and your new event hub is completed.
After configuring your new Event Hub, follow the appropriate Splunk app instructions on ingesting the data.
If you need assistance in this new configuration or have questions about what TekStream can do for you, please reach out to us.
About the Author
Kyle Moreau has over 8 years of information technology experience, 6 of which are within information security. As a security engineer, he leads projects for various security tools including SIEM and Endpoint. Maintained and developed solutions to improve the security posture of a fortune 500 organization. Kyle has successfully completed full ES implementations for multiple organizations. Previous experience as a security analyst, focused on incident response and investigation as well as keeping current in the latest attack methods.