How to Forward Data to Splunk Cloud: Architecture Options and Step-by-Step Instructions

      By: Forrest Lybarger & Khristian Pena  |  Splunk Consultants

Implementing Splunk Cloud prompts teams to make many decisions about their environment, from hardware specs to compliance standards. One of the big questions a team must answer is, “How will data be sent from devices like workstations and domain controllers to Splunk Cloud?” But that is more complicated than it may seem. Besides the niche forwarding methods (i.e. Splunk Stream App, which usually is mandatory for whatever data will use it), there are 3 options for forwarding data: directly via universal forwarder (UF), indirectly via intermediate forwarder (IF), or directly via a heavy forwarder (HF). Each of these methods has pros and cons that will be covered here, so anyone moving to Splunk Cloud can make a decision on how they will forward data. These strategies also aren’t mutually exclusive; they can be mixed and matched depending on individual circumstances.

Option 1: Send Data to the Splunk Cloud via a Universal Forwarder

A flowchart with three tan boxes connected to a brick firewall icon. The firewall icon is connected to a cloud icon.

The first and simplest option is to send data directly from source hosts to Splunk Cloud via a UF. This approach doesn’t require any additional hardware (unless a deployment server is used) and has no single point of failure. UFs are installed on every source host and are configured with the environment-specific Splunk Cloud Forwarding app (downloadable from every Splunk Cloud Web UI). Problems emerge when considering the connection between the source hosts and Splunk Cloud. Firewall rules will need to be mended in order to allow outbound traffic from source hosts. With a small environment, this ask is easy to implement and maintain, but once the environment scales up, there can be thousands of firewall rules to maintain. The cut off for when this option is viable will depend on the customer, but in general this set up is best for small environments.

Option 2: Send data to the Splunk Cloud via an Intermediate Forwarder

A flowchart depicting how an intermediate forwarder can be used to send data to the Splunk Cloud.

The second option is to send data to an intermediate forwarder before sending to Splunk Cloud. The intermediate forwarder will need to be on its own server in order to have the resources for processing large amounts of data. It is highly recommended to have at least two IFs to prevent a single point of failure (Splunk can load balance between IFs without a dedicated load balancer). IFs use the same software as UFs, so they are very lightweight and perform minimal processing. The main benefit of this architecture is to minimize firewall holes. With this approach, only the IFs will need special firewall rules maintained for them. You can also send heavy forwarder data through these IFs, though that will increase network load due to the increased size of parsed data. Another consideration with this plan is data transmission between data centers. If such a thing is not allowed, there will need to be IFs in every datacenter where there are UFs, and planning must happen to send data to the correct IF. Overall, this option is more scalable than the first, but requires more hardware and coordination.

Option 3: Send Data to the Splunk Cloud via a Heavy Forwarder

A flowchart depicting how a heavy forwarder can be used to send data to the Splunk Cloud.

Finally, data can be sent to a heavy forwarder (HF) before going to Splunk Cloud. Similar to the previous option, the HF acts as an intermediary that requires its own hardware. The big change from the previous architecture is that the HF is a full Splunk instance and parses data sent through it. Parsed data will be significantly larger than unparsed data causing more network load, but will reduce load on indexers in Splunk Cloud and is necessary for some data (individual data source documentation will note if HFs are required). Two or more HFs are recommended to prevent a single point of failure. Like the previous option, this is more scalable than the first option, but requires more hardware and coordination.

In conclusion, the three architectures have their own purposes and can be used in tandem to fulfill each customer’s specific needs. For small environments where firewall management isn’t a problem, sending data directly from source hosts to Splunk Cloud is a viable option with limited extra expenditure. For larger environments, a mix of the other two methods is best. When there’s data that needs an HF, use the third approach. Use the second approach for other data to avoid excessive firewall rules. Whether or not to forward the HF data through an IF is up to the individuals. Doing so will use more network bandwidth, but sending to Splunk Cloud from the HF will require more firewall rules.

Forwarder Configuration to Splunk Cloud

Regardless of which architecture your organization decides to go with for sending local data to Splunk Cloud, you will need to install the Splunk Universal Forwarder software from your Cloud Deployment. The package is called Splunk Universal Forwarder Credentials, but you are able to install this app on your UFs as well as your HFs.

The Universal Forwarder Credentials file contains a custom certificate for your Splunk Cloud Deployment.

Download the Splunk Universal Forwarder Credentials:

  1. In your Splunk Cloud instance, got to Apps > Universal Forwarder.
  2. Click Download Universal Forwarder Credentials.
  3. Note the location of the downloaded file; it will be named splunkclouduf.spl.
  4. Copy the file to your /tmp folder on the instance that will be receiving the credentials or to your Deployment server.

A screenshot of the Splunk Cloud app illustrating where the Universal Forwarder Credentials can be accessed and downloaded.

Install the forwarder credentials on individual forwarders.

(We recommend managing all forwarders from your Deployment Server vs. manually updating each instance.)

This will only apply if you do not use a Deployment Server to manage your forwarders.

  1. Install the following app on your forwarder by running this command: /opt/splunkforwarder/bin/splunk install app /tmp/splunclouduf.spl
  2. When you are prompted for a login, use the user name and password for the Universal Forwarder instance. The following message will display when you have successfully installed the credentials package: App ‘/tmp/splunkcloud.spl’ installed.
  3. Restart the forwarder: /opt/splunkforwarder/bin/splunk restart

Want to learn more about forwarding data to the Splunk Cloud? Contact us today!

Press Release: TekStream Makes INC. 5000 List for Seventh Consecutive Year

For the 7th Time, Atlanta-based Technology Company Named One of the Fastest-growing Private Companies in America 

Atlanta-based technology company, TekStream, is excited to announce that for the seventh time in a row, it has made the Inc. 5000 list of the fastest-growing private companies in America. Only 3% of companies have made the list seven times. This prestigious recognition comes again ten years after Rob Jansen, Judd Robins, and Mark Gannon left major firms and pursued a dream of creating a strategic offering to provide enterprise technology software, services, solutions, and sourcing. Now, they’re a part of an elite group that, over the years, has included companies such as Microsoft, Timberland, Vizio, Intuit, Chobani, Oracle, and Zappos.com. 

“Being included in the Inc. 5000 for the seventh straight year is something we are truly proud of as less than 3% of the organizations ever making the list have made it seven times in the history of the Inc. 5000,” said Chief Executive Officer, Rob Jansen. “Continued adoption by our clients, especially with the impact of the COVID-19 pandemic, for cloud-based technologies, Security, and Big Data solutions to solve complex business problems has been truly remarkable. We are helping our clients take advantage of today’s most advanced recruiting and technology solutions to digitally transform their businesses and address the ever-changing market.” 

Not only have the companies on the 2021 Inc. 5000 been very competitive within their markets, but this year’s list also proved especially resilient and flexible given 2020’s unprecedented challenges. Among the 5,000, the average median three-year growth rate soared to 543 percent, and median revenue reached $11.1 million. Together, those companies added more than 610,000 jobs over the past three years. 

“This past year has introduced numerous operational challenges for everyone, and our customers are certainly no exception. The remote nature of the “new normal” for digital transformation support and collaboration has put all IT firms to the test. To make the INC. 5000 list again during this period validates our ability to adapt quickly to new processes, our resiliency to make our customers successful regardless of the barriers, but most importantly the unwavering resolve of our family culture to stick together and get the job done” said Executive Vice President, Judd Robins. 

To qualify for the award, companies had to be privately owned, established in the first quarter of 2015 or earlier, experienced a two-year growth in sales of more than 50 percent, and garnered revenue between $2 million and $300 million in 2020. 

“With the unprecedented challenges of 2020, we are honored to receive this accolade once again. It is evidence of our team’s dedication and the many client and candidate relationships we have built and service daily. We continue to adapt our RPO and Contingent Recruiting solutions to meet changing conditions and look forward to earning additional recognition in the future” said Executive Vice President of Talent Management and Recruiting Services, Mark Gannon. 

TekStream accelerates clients’ digital transformation by navigating complex technology environments with a combination of technical expertise and staffing solutions. We guide clients’ decisions, quickly implement the right technologies with the right people, and keep them running for sustainable growth. Our battle-tested processes and methodology help companies with legacy systems get to the cloud faster, so they can be agile, reduce costs, and improve operational efficiencies. And with 100s of deployments under our belt, we can guarantee on-time and on-budget project delivery. That’s why 97% of clients are repeat customers. For more information visit https://www.tekstream.com/