A Detailed Guide to using the get_list() function in Splunk SOAR
By David Burns, Senior SOAR Engineer
When working with Splunk SOAR, many users are familiar with the documentation for the get_list() function. This function retrieves a list of lists based on the list_name parameter. However, the real power of get_list() emerges when you start incorporating optional parameters. In this guide, we’ll walk through the usage of these optional parameters, highlight common error messages, and explore best practices for debugging your API calls.
Understanding the get_list() function
The get_list() function is commonly used in Splunk SOAR to fetch a list of lists, where each element in the returned list corresponds to a row of data from the specified list. The primary parameter is the list_name, which specifies which list to retrieve. However, as with any function, understanding how optional parameters affect the output is crucial for effective use.
We’ll take a closer look at the optional parameters and the scenarios where they might cause issues, such as error messages or unexpected behavior. We’ll also cover how to enable additional debugging information by turning on trace logging for deeper insights.
Exploring Optional Parameters for get_list()
1. The column_index Parameter
One of the optional parameters you can use with get_list() is column_index. This parameter allows you to narrow down the rows returned by specifying a column index to search within.
However, it’s important to note that the column_index parameter does not work as expected. This has been confirmed in Splunk SOAR version 6.3.1.178, but similar issues have been observed in all versions of 6.x. Here’s what we found during testing:
- Data Validation: There is no data validation for the column_index parameter. We tested this by passing strings and other objects into the column_index parameter, and the function did not fail.
- Out-of-Bounds Values: Testing values that fall outside the potential columns (both positive and negative directions) did not trigger an error either.
- Trace Logging: Turning on trace logging did not provide any additional insights into the root cause of this issue.
Given the lack of validation and behavior across versions, it’s important to use this parameter cautiously and keep an eye on potential inconsistencies.
2. The values Parameter
The values parameter allows you to specify which values you are searching for within the list. This parameter can accept either a single string or a list of strings. Understanding how this parameter behaves is critical for troubleshooting.
Using values in a Custom Function
When you try to use the values parameter inside a custom function, you may encounter the following error:
Dec 27, 13:47:20 : RuntimeError: phantom.datastore_present() cannot be called from within a custom function
Dec 27, 13:47:20 : Unexpected error in get_list(): RuntimeError: phantom.datastore_present() cannot be called from within a custom function
Dec 27, 13:47:20 : phantom.get_list results: success: False, message: Unexpected error, check debug output, execs: []
This error occurs because the phantom.datastore_present() function cannot be called inside a custom function. However, values works when used in a custom code block. This is an important distinction to make, as it means you can use the values parameter outside of custom functions for successful results.
3. Structure of the Data Returned by get_list()
Understanding the structure of the data returned by the get_list() function is essential for effective use. The standard use of the function returns a list of lists. However, when using the values parameter:
- Index: Refers to the row where the value was found.
- Value: The key that contains the entire row of data that includes the found value.
Example: Expected Output Structure
When you query get_list() with the values parameter of Doe, you’ll receive results structured like this:
{
"status": "success",
"data": [
{
"index": 0,
"value": ["John", "Doe", "30"]
},
{
"index": 1,
"value": ["Jane", "Doe", "28"]
}
]
}· The index references the row (e.g., row 0, row 1).
· The value contains the entire row that includes the matching value.
How the values Parameter Works: String vs. List
As the documentation states, the values parameter can accept a string or a list of strings. The behavior differs depending on whether you pass a single value or a list:
- Single Value: When you pass a string (e.g., “A”), get_list() will return any row where any column contains an exact match for “A”.
- List of Values: When you pass a list of strings (e.g., [“A”, “B”]), the results will be the union of the values, not the intersection. This means that any row containing either “A” or “B” in any column will be included in the results.
Example of values with a List:
Let’s say you use the following list as your values parameter: [“A”, “B”].
phantom.get_list(list_name="my_list", values=["A", "B"])The returned results will include any row that has either “A” or “B” in any column. It’s a union of the two values, not an intersection.
Example Scenario:
Consider the following list of values:
| Column1 | Column2 | Column3 |
| A | C | D |
| B | E | F |
| G | H | I |
When you query get_list() with values=[“A”, “B”], you will receive the following results:
{
"status": "success",
"data": [
{
"index": 0,
"value": ["A", "C", "D"]
},
{
"index": 1,
"value": ["B", "E", "F"]
}
]
}Rows 0 and 1 are returned because they contain either “A” or “B” in any column, not necessarily in the same column.
Use Case: Automating the Search for Matching Rows
One of the key strengths of the get_list() function is its ability to automate the search for matching rows within a large dataset. Instead of manually sifting through large lists, you can leverage Splunk SOAR to automatically find the rows that match certain criteria.
Scenario: Matching ASN Values or Tenancy Information
Let’s consider a scenario where you have a large list of Autonomous System Numbers (ASNs) or Tenancy information. In such cases, you may need to identify rows that contain similar or matching values to help with threat detection, incident response, or reporting.
For example, imagine you have a list of ASNs associated with various IP addresses:
| ASN | IP Address | Owner |
| 12345 | 192.168.1.1 | Company A |
| 67890 | 192.168.1.2 | Company B |
| 12345 | 192.168.1.3 | Company C |
| 54321 | 192.168.1.4 | Company D |
If you’re interested in finding all rows that contain ASN 12345, instead of manually reviewing each row, you can use get_list() with the values parameter. Here’s how you can automate the process:
phantom.get_list(list_name="asn_list", values=["12345"])
This query will return any row that contains ASN 12345 in the ASN column, allowing you to quickly retrieve all related entries.
You can apply this same method to other use cases, such as filtering rows based on tenancy information, user IDs, or IP address ranges, helping you quickly identify patterns or potential security issues. By using SOAR to find the correct rows, you reduce the probability of problems in your own implementation.
Error Handling and Debugging Tips
When working with get_list(), it’s important to handle errors effectively and understand the debug output. Here’s how you can troubleshoot common issues:
- Check Trace Logs: Always enable trace logging to get more detailed debug information. This can provide insight into issues like unexpected errors or misbehaving parameters.
- Validate Parameters: Ensure that the parameters you are passing (e.g., column_index, values) are valid for the version of Splunk SOAR you’re using.
- Use Custom Code Blocks: If you’re encountering issues with custom functions (e.g., phantom.datastore_present() errors), move your logic into a custom code block instead.
Conclusion: Mastering the get_list() Function in Splunk SOAR
Understanding how to use the get_list() function and its optional parameters is crucial for automating tasks and querying data effectively in Splunk SOAR. While the function works well in most cases, there are a few nuances to be aware of, such as issues with column_index and errors when using values inside custom functions.
By following this guide and using the provided examples, you should be well-equipped to leverage get_list() for your own automation and security orchestration tasks.
About the Author
David Burns is a security engineer with experience working with Splunk Enterprise Security and Splunk SOAR (formerly Phantom) for a large fortune 200 bank. Before that he was a System Security Engineer working on the automation of security testing of OT systems.
He brings his 20+ years programming background to use SDLC in rapid development of playbooks, custom functions, and more leading to modularity, re-use in design, and better long-term maintenance. For example, creating deeper integration for escalation through Slack and creating EDL management for multiple clients.
At Tekstream, he developed the slack escalation methodology that notifies customers of events that need their attention as well as a way of process for generating and updating EDLs within Splunk.