Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Incident AI
description: "Learn how to use Incident AI to automate incident coordination, generate summaries, send notifications, and create AI-assisted postmortems."
description: "Learn how to use Incident AI to investigate active incidents, receive proactive summaries, detect related issues, and trigger AI-powered root cause analysis."
aliases:
- /bits_ai/managing_incidents/
- /bits_ai/bits_ai_sre/coordinate_incidents
Expand All @@ -10,72 +10,23 @@ further_reading:
- link: "/bits_ai/bits_ai_sre/"
tag: "Documentation"
text: "Learn about Bits AI SRE"
- link: "/incident_response/incident_management/post_incident/postmortems"
tag: "Documentation"
text: "Generate AI-assisted postmortems"
- link: "/incident_response/incident_management/setup_and_configuration/templates"
tag: "Documentation"
text: "Configure AI variables in notification and postmortem templates"
---

## Overview

Incident AI transforms how your team manages incidents by automating coordination tasks and providing intelligent insights throughout the incident lifecycle. Built into Datadog Incident Management, it works in Slack and the Datadog platform to help you respond faster and learn from every incident.

Key capabilities include:
- **AI-driven incident coordination**: Use conversational prompts in Slack to declare incidents, update status and severity, search incident history, and receive next-step recommendations throughout the incident life cycle.
- **Automated insights and summaries**: Instantly receive incident summaries, AI-generated detection of related and recurring issues, and actionable contributing factors—directly in Slack and Datadog.
- **AI-powered stakeholder updates and postmortems**: Dynamically populate notifications across email, MS Teams, Slack, and more with AI-generated summaries, and generate comprehensive postmortem drafts with executive summaries, impact analysis, timelines, and lessons learned.

## Get started with incident coordination

Incident AI helps coordinate incidents—especially those involving multiple teams—by suggesting next steps throughout the incident lifecycle. This streamlines communication and improves overall process management.

1. Connect Datadog to Slack.
1. In any Slack channel, run the `/dd connect` command.
1. Follow the on-screen prompts to complete the connection process.
1. Enable the Slack integration in Datadog Incident Management.
1. In the [Integrations][4] section of the Incidents settings page, find the **Slack** settings.
1. Enable the following toggles:
- **Push Slack channel messages to the incident timeline**
- **Activate Incident AI features in incident Slack channels for your organization**<br />
**Note**: Incident AI's incident management features can only be activated for one Datadog organization within a single Slack workspace.
1. [Chat with Incident AI](#chat-with-incident-ai) in a Slack channel, invite it by running the `@Datadog` command.
1. [Start a Bits AI SRE investigation](#bits-ai-investigation).

## Customize stakeholder notifications

Incident AI can dynamically populate key details in stakeholder notifications, delivering clearer, faster updates across the tools your team already uses. Notification rules support delivery to a wide variety of destinations, including email, Datadog On-Call, MS Teams, Slack, and more, ensuring AI-enhanced updates reach the right people, on the right platform, at the right time.

1. In your Incidents settings, go to [Notification Templates][1].
1. Create a new template or edit an existing one.
1. In the message body, insert any of the following AI variables:
<table>
<thead>
<tr>
<th>Field</th>
<th>Variable</th>
</tr>
</thead>
<tbody>
<tr>
<td>AI Contributing Factors</td>
<td><code>{{incident.ai_contributing_factors}}</code></td>
</tr>
<tr>
<td>AI Impact</td>
<td><code>{{incident.ai_impact}}</code></td>
</tr>
<tr>
<td>AI Issue</td>
<td><code>{{incident.ai_issue}}</code></td>
</tr>
<tr>
<td>AI Remediation</td>
<td><code>{{incident.ai_remediation}}</code></td>
</tr>
</tbody>
</table>
{{< img src="service_management/incidents/incident_ai/message_template_variables.png" alt="New message template with AI variables in it" style="width:100%;" >}}
1. Click **Save** to save the template.
1. Go to your incident [Notification Rules][2].
1. Click **New Rule**.
1. Under **With template...**, select the message template you just created.
1. Click **Save** to save the notification rule.
Incident AI helps responders investigate active incidents faster by automating information gathering and surfacing relevant context. Working in Slack and the Datadog platform, it provides proactive summaries, detects related incidents, and enables conversational queries against incident history so your team can focus on resolution.

To enable Incident AI, connect Datadog to Slack and activate Bits AI in your incident Slack channels. For setup instructions, see the [Slack integration for Incident Management][1].

For AI features that apply outside the investigation phase, see:
- [Templates][2]: Add AI variables to notification and postmortem templates
- [Postmortems][3]: Generate AI-assisted postmortem drafts

## Proactive incident summaries

Expand All @@ -85,14 +36,6 @@ When an incident is changed to resolved, Incident AI posts a final summary. This

{{< img src="service_management/incidents/incident_ai/incident_summary.png" alt="Example incident summary in Slack" style="width:100%;" >}}

## Proactive follow-up task suggestion

After an incident is resolved, Incident AI collects any follow-up tasks responders mentioned during the incident. It then prompts you to review and create them with a single click. These tasks are saved as Incident Follow-Ups in Datadog Incident Management. For more information, see [Incident Follow-ups][5].

To view suggested follow-up tasks:
1. Navigate to the relevant incident in Datadog.
1. Open the **Post-Incident** tab to view a list of all follow-up tasks you've saved from Slack.

## Related incident detection

Incident AI automatically flags related incidents if they are declared within 20 minutes of each other, helping you identify broader systemic issues.
Expand Down Expand Up @@ -129,52 +72,19 @@ When triggered, the agent automatically pulls in the incident's timeline, linked

{{< img src="incident_response/incident_management/incident_ai/Triggering_investigations_slack.png" alt="A Datadog incident Slack channel showing a completed Bits AI SRE investigation with root cause findings and a View Full Investigation button" style="width:80%;" >}}

**From the web UI**, you can trigger an investigation directly from the "Investigation" section on the incident overview page.
**From the web UI**, you can trigger an investigation directly from the "Investigation" section on the incident overview page.

{{< img src="incident_response/incident_management/incident_ai/Triggering_investigations_web.png" alt="The Incident Management overview page showing a completed Bits AI SRE investigation in the Investigation section with a View Full Investigation button" >}}

When triggered, the investigation is embedded in the incident web UI, and Bits AI SRE appears as a responder in the Responder Roles section of the page.


For setup instructions and prerequisites, see the [Slack Integration][7] documentation. For a full walkthrough of the investigation workflow, see [Bits AI SRE][8].

## Customize postmortem templates with AI incident variables

1. In Datadog, navigate to your incident [Postmortem Templates][3].
1. Click **New Postmortem Template**.
1. Customize your template using the following AI variables for dynamic AI-generated content:
| Description | Variable |
|-------------------------------------|-----------------------------------|
| Executive summary | `{{incident.ai_summary}}` |
| System context and dependencies | `{{incident.ai_system_overview}}` |
| Key event timeline | `{{incident.ai_key_timeline}}` |
| Summary of customer impact | `{{incident.ai_customer_impact}}` |
| Follow-up actions | `{{incident.ai_action_items}}` |
| Key takeaways for future prevention | `{{incident.ai_lessons_learned}}` |
<p><strong>Note</strong>: AI variables must be preceded by a section header.</p>
1. Click **Save**. Your new template appears as a template option during postmortem generation.

## Generate a first draft of the incident postmortem

To generate an AI-assisted postmortem draft:
1. In Datadog, navigate to the resolved incident you'd like to generate a postmortem for.
1. Ensure the incident timeline contains at least 10 messages.
1. Click **Generate Postmortem**.
1. Under **Choose Template**, select either the out-of-the-box **General incident with AI content** template, or a custom template that you've created.
1. Click **Generate**. Allow up to one minute for the postmortem to be generated. Do not close the tab during this time.
1. Review the AI-generated postmortem draft. It serves as a starting point for your incident responders. Datadog recommends reviewing and refining the draft before sharing it.

For more information on generating and managing postmortems, see [Incident Postmortems][6].
For a full walkthrough of the investigation workflow, see [Bits AI SRE][4].

## Further reading

{{< partial name="whats-next/whats-next.html" >}}

[1]: https://app.datadoghq.com/incidents/settings?section=message-templates
[2]: https://app.datadoghq.com/incidents/settings?section=notification-rules
[3]: https://app.datadoghq.com/incidents/settings?section=postmortem-templates
[4]: https://app.datadoghq.com/incidents/settings?section=integrations
[5]: /incident_response/incident_management/post_incident/follow-ups
[6]: /incident_response/incident_management/post_incident/postmortems
[7]: /integrations/slack/?tab=datadogforslack
[8]: /bits_ai/bits_ai_sre
[1]: /incident_response/incident_management/setup_and_configuration/integrations/slack
[2]: /incident_response/incident_management/setup_and_configuration/templates
[3]: /incident_response/incident_management/post_incident/postmortems
[4]: /bits_ai/bits_ai_sre
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ Common examples for creating follow-ups include:

By capturing these items as follow-ups, your team can stay focused on incident resolution while ensuring important improvements aren't forgotten.

## AI-suggested follow-up tasks

After an incident is resolved, Incident AI scans the incident channel for follow-up tasks that responders mentioned during the incident. It then prompts you to review and create them with a single click. Tasks saved this way appear as Incident Follow-ups in Datadog Incident Management.

To view AI-suggested follow-up tasks:
1. Navigate to the relevant incident in Datadog.
1. Open the **Post-Incident** tab to view a list of all follow-up tasks saved from Slack.

## Create and manage follow-ups

Follow-ups can be created at any point during an incident (even before it's resolved), allowing responders to document necessary work as they discover it. After resolution, you can [export follow-ups](#export-follow-ups) to Jira or Case Management to integrate them into your team's existing workflows.
Expand Down Expand Up @@ -64,7 +72,6 @@ After defining an export template, you have two options:

### Case Management exports


When you export your follow-ups to [Case Management][2], you can manage, track, and analyze your follow-ups directly in Datadog. For example, you can:

* View all of the open follow-up cases assigned to a particular user in Datadog
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,8 @@ Use the **Save Location** dropdown in the template editor to select where the ge

Templates support variables that populate with incident data when the postmortem is generated. Variables allow you to include relevant incident context automatically without copying it manually. For a full list of available variables, see the [Incident Variables][6] reference.

**Standard variables** pull directly from incident fields:
{{% collapse-content title="Standard variables" level="h4" %}}
Standard variables pull directly from incident fields and also include any defined [custom property fields][5].

| Variable | Description |
|---|---|
Expand All @@ -77,12 +78,38 @@ Templates support variables that populate with incident data when the postmortem
| `{{incident.severity}}` | The incident severity level |
| `{{incident.commander}}` | The incident commander's name, or their email or handle if no name is set |

Any [custom property fields][5] you have defined are also available as variables.
{{% /collapse-content %}}

{{% collapse-content title="Incident card variable" level="h4" %}}
The incident card variable (`{{incident.card}}`) embeds a summary card that reflects incident fields in real time. When incident fields change, the card in the postmortem updates automatically.

**The incident card variable** (`{{incident.card}}`) embeds a summary card that reflects incident fields in real time. When incident fields change, the card in the postmortem updates automatically, reducing the need to switch between the two.
**Note**: The incident card is only supported in Datadog Notebooks.
{{% /collapse-content %}}

{{% collapse-content title="AI variables" level="h4" %}}
AI variables use Bits AI to generate suggested content based on incident data. When you generate a postmortem that includes an AI variable, Datadog provides a suggested value that you can accept, edit, or reject before it is saved.

| Variable | Description |
|---|---|
| `{{incident.ai_summary}}` | Executive summary |
| `{{incident.ai_system_overview}}` | System context and dependencies |
| `{{incident.ai_key_timeline}}` | Key event timeline |
| `{{incident.ai_customer_impact}}` | Summary of customer impact |
| `{{incident.ai_action_items}}` | Follow-up actions |
| `{{incident.ai_lessons_learned}}` | Key takeaways for future prevention |

To use these variables:
1. Create or edit a postmortem template in [**Incident Management Settings > Post-Mortem Templates**][4].
1. Add the variables to the template body.
1. When generating a postmortem, select the **General incident with AI content** out-of-the-box template, or a custom template that includes AI variables.
1. Allow up to one minute for AI-generated content to populate. Datadog recommends reviewing and refining AI-generated content before sharing the postmortem.

**Note**:
- AI variables must be preceded by a section header.
- The incident timeline must contain at least 10 messages for AI variables to generate content.


**AI variables** use Bits AI to generate suggested content based on incident data. When you generate a postmortem that includes an AI variable, Datadog provides a suggested value that you can accept, edit, or reject before it is saved. The `{{incident.ai_summary}}` variable, for example, generates a suggested summary of what occurred during the incident.
{{% /collapse-content %}}

## Attach an existing postmortem

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,19 @@ To create a message template:

**Note:** Template variables are supported in both the message's title and body.

### AI variables for notifications

Incident AI can dynamically populate key details in notifications with AI-generated content. Insert any of the following variables into your message body:

| Field | Variable |
|---|---|
| AI Contributing Factors | `{{incident.ai_contributing_factors}}` |
| AI Impact | `{{incident.ai_impact}}` |
| AI Issue | `{{incident.ai_issue}}` |
| AI Remediation | `{{incident.ai_remediation}}` |

{{< img src="service_management/incidents/incident_ai/message_template_variables.png" alt="New message template with AI variables in it" style="width:80%;" >}}

## Postmortems

Postmortem templates are dynamic, reusable templates used to create a postmortem that is automatically populated with incident information after an incident has been resolved. Postmortem templates use [template variables][4], such as `{{incident.severity}}`, to dynamically inject the corresponding value from the incident that the postmortem is being created for.
Expand All @@ -46,6 +59,21 @@ To create a postmortem template:
5. (Optional) Set the template as the default.
6. Click **Save**.

### AI variables for postmortems

Postmortem templates support AI variables that use Bits AI to generate suggested content based on incident data. When you generate a postmortem that includes an AI variable, Datadog provides a suggested value to accept, edit, or reject.

**Note**: AI variables must be preceded by a section header.

| Variable | Description |
|---|---|
| `{{incident.ai_summary}}` | Executive summary |
| `{{incident.ai_system_overview}}` | System context and dependencies |
| `{{incident.ai_key_timeline}}` | Key event timeline |
| `{{incident.ai_customer_impact}}` | Summary of customer impact |
| `{{incident.ai_action_items}}` | Follow-up actions |
| `{{incident.ai_lessons_learned}}` | Key takeaways for future prevention |

For more information on generating postmortems using these templates, see [Incident Postmortems][5].

## Further reading
Expand Down
Loading