Link

Receive Signals from Datadog Triggered Monitors

Integration Overview

  1. Create an API Key in Datadog.
  2. Create a Datadog Integration in Zebrium using the information from step 1.
  3. Create a Webhook Integration in Datadog using the information from step 2.
  4. Add Webhook notifications to your Triggered Monitors in Datadog.
  5. Add Zebrium Root Cause Reports to your Datadog Dashboard.

Integration Details

STEP 1: Create an API Key in Datadog

  1. From the Main Navigation panel, hover over your Datadog Login Name and select Organization Settings.
  2. Click on API Keys.
  3. Click the + New Key button.
  4. Enter a Name for the API Key and click Create Key.
  5. Copy and save the Key for use in STEP 2.

STEP 2: Create a Datadog Integration in Zebrium to Receive Signals from Datadog

  1. From the User menu area in Zebrium, click on the Settings (hamburger) Menu.
  2. Select Integrations.
  3. Scroll to the Observability Dashboards section and click on Datadog Events and Metrics.
  4. Click on the Create a New Integration button.
  5. Click on the General tab.
  6. Enter an Integration Name for this integration.
  7. Select the Deployment for the integration.
  8. Select the Service Group(s) for the integration.
  9. Click on the Receive Signals tab.
  10. Click on the Enabled button.
  11. Enter the API Key created in STEP 1 above.
  12. Click the Save button.
  13. Copy the Webhook URL and save for use in STEP 3 and click OK.

STEP 3: Create a Webhook Integration in Datadog

  1. From the Main Navigation panel, navigate to Integrations/Integrations.
  2. Locate the Webhooks integration card and click Configure.
  3. Click on the New button located in the Webhooks **section and enter a **Name and the URL saved in STEP 2.
  4. In the Payload section, add: "alert_transition": "$ALERT_TRANSITION" after "event_type": "$EVENT_TYPE",
  5. Click Save.

STEP 4: Add Webhook notifications to your Triggered Monitors in Datadog

  1. From the Main Navigation panel, navigate to Monitors/Manage Monitors.
  2. Click on the Monitor you wish to trigger Root Cause Reports.
  3. Choose Edit from the gear icon on the Monitor page.
  4. Add the webhook (from STEP 3) in the Notify your team list.
  5. Click Save.

STEP 5: Add Zebrium Root Cause Report Detections to any of your Datadog Dashboards

Zebrum sends events to Datadog as follows:

  1. Events - events are sent each time a Zebrium Root Cause Report Detection occurs.

Visualizing Zebrium Data in Datadog

Here is a sample Chart visualization showing:

  1. A Root Cause Finder panel that displays a vertical bar whenever a Zebrium detection occurs. This allows you to easily see detections that are aligned with other metrics on your dashboards.
  2. A Root Cause Reports Summary panel that list summary information for each Zebrium detection.

Here is a screen shot showing the definition of the Root Cause Finder Panel:

Here is a screen shot showing the definition of the Root Cause Reports Summary Panel:

Table of Important Metric Names

Metric Name Description
zebrium.logs.all.count Count of all log events received in a one minute duration (per service_group and deployment)
zebrium.logs.anomalies.count Count of anomaly log events received in a one minute duration (per service_group and deployment)
zebrium.logs.errors.count Count of error log events received in a one minute duration (per service_group and deployment)
ze_service_group Zebrium service group name for the corresponding metric or event
ze_deployment Zebrium deployment name for the corresponding metric or event
ze_significance Significance of the Root Cause Report (low, medium or high)

Support

If you need help with this integration, please contact Zebrium by sending email to support@zebrium.com.