Here at Function1 we use Slack in order to stay in constant contact with our co-workers. If you haven't heard of Slack before, Slack is a team chat and communication tool. We use it to talk about our projects, company announcements, sports, random water cooler talk, technical questions, etc. Slack has integration built-in with a lot of services.

• Slack is a common way to communicate with other Splunk users outside of user groups, the annual conference (.conf), or other Splunk and industry events. This form of chat enables anyone to communicate with the greater Splunk Community—customers, partners, and Splunkers—worldwide.
• Use a webhook alert action. Webhooks allow you to define custom callbacks on a particular web resource. For instance, you can set up a webhook to make an alert message pop up in a chat room or post a notification on a web page.
• This Tech Talk is a step by step tutorial on how to ingest your Slack data in Splunk, through the Splunk Add-on for Slack, which is an add-on that leverages the Slack Audit logs API to gain additional insight into your organization's security posture.
• The Investor Relations website contains information about Splunk Inc.' S business for stockholders, potential investors, and financial analysts.

For those using log tools such as Splunk, you can setup alerts. These will run queries every so often and trigger actions when conditions are met e.g. count of events surpasses a threshold.

This post is about pushing those alerts from Splunk to Slack. When an alarm is triggered, a JSON request is made by Splunk to a URL you provide. This gives the benefit of your own customisation of messages.

Reasons

You can find applications on the Splunk app store to post alerts to Slack. However you may have issues when running such apps on a cluster. Or you may have permission issues in an enterprise environment. In my situation, I ran into both problems.

Flow

1. Splunk invokes an AWS API Gateway endpoint.
2. The AWS API Gateway endpoint invokes an AWS Lambda function.
3. The AWS Lambda function sends a message to a Slack inbound web hook (and delivers the message your own desired Slack channel).

Step 1 - Create Slack Inbound Webhook

Go to the following page, whilst logged-in to your Slack workspace:

Or alternatively, from Slack’s website:

• Configure apps
• Custom integrations (sidebar)
• Incoming WebHooks

More information about inbound web hooks can be found here:

Once you’ve setup an inbound webhook, copy the Webhook URL for the next step. Let’s pretend it’s:

Step 2 - Setup AWS Lambda Function

Create a new Lambda function from scratch:

For this example, we’ll call it splunk alert.

Then paste in the following, but edit the request options with parts of the Webhook URL from the previous step:

Towards the end is the JSON object message Download microsoft word free for macbook. , which is the Slack message written to the Slack inbound webhook. This can be heavily customised.

Docs on Slack message format:

Once you’re happy with your Lambda, publish it. At the top, go to Actions and select Publish new version.

Step 3 - Create AWS API Gateway Trigger

On your AWS Lambda function page, go to the Triggers tab and add a trigger.

You will then see a dotted box, click it and select API Gateway.

Then will in the form with an API name (such as splunk-alerts) and set the Security to Open (although you may want to change this later):

You should now have API Gateway available as a trigger. Click the arrow icon to show the endpoint’s URL.

Step 4 - Test Trigger

Splunk Slack Webhook Alert

Let’s say the trigger URL is:

Just make a POST request with the following test payload:

In Chrome I use the Postman app, but this is simple enough to achieve with cURL as well:

Step 5 - Splunk Alert

Run a query on the Splunk search application. Once it has finished loading, select Save As and then Alert.

Configure the alert as needed; useful docs:

After the alert is setup, add a Webhook Download facetime for macbook air. action and set the URL to the endpoint created earlier.

Splunk Slack Integration

Summary

Splunk Slack Alert

You should now have monkeyboy to save the day: