Skip to content

Incident Webhooks

GitPulse provides a webhook endpoint to receive incident notifications from external monitoring systems. This allows you to track incidents and calculate Mean Time to Recovery (MTTR) metrics automatically.

Overview

The incident webhook endpoint accepts POST requests with incident data and stores them in GitPulse for analysis. This enables automatic incident tracking without manual data entry.

Webhook Endpoint

URL: /api/incidents/webhook/

Method: POST

Content-Type: application/json

Authentication

⚠️ API Key Required: The webhook endpoint requires API key authentication for security and CSRF protection.

API Key Configuration

  1. Generate an API key in your GitPulse project settings
  2. Include the API key in the request header: X-API-Key: <your-api-key>

Example Request Headers

POST /api/incidents/webhook/ HTTP/1.1
Host: your-gitpulse-instance.com
Content-Type: application/json
X-API-Key: your-api-key-here

Request Format

The webhook expects a JSON payload with the following structure:

{
  "incident_id": "unique-incident-identifier",
  "project_uuid": "gitpulse-project-uuid",
  "status": "ALERT|RECOVERY",
  "incident_time": "2025-01-15T10:30:00Z",
  "monitor_name": "Database Connection",
  "monitor_id": "db-conn-001",
  "repo_uuid": "gitpulse-repo-uuid",
  "message": "Database connection timeout after 30 seconds"
}

How Status Mapping Works

GitPulse automatically maps monitoring tool statuses to internal ALERT and RECOVERY values for consistent incident processing and MTTR calculations.

Automatic Status Conversion

  1. ALERT Statuses: Any status indicating a problem (e.g., DOWN, FIRING, CRITICAL) is converted to ALERT
  2. RECOVERY Statuses: Any status indicating resolution (e.g., UP, RESOLVED, OK) is converted to RECOVERY
  3. Unknown Statuses: Unrecognized statuses are kept as-is for flexibility

Benefits of Status Mapping

  • Tool Agnostic: Use any monitoring tool's native status values
  • Consistent Processing: All incidents are processed uniformly regardless of source
  • Automatic MTTR: Recovery events automatically calculate duration from previous alerts
  • Flexible Integration: No need to modify existing monitoring configurations

How incident_id Works

The incident_id field is designed to be reusable for the same project, allowing you to track multiple events (like DOWN/UP cycles) for the same incident.

Usage Patterns

  • Single Incident, Multiple Events: Use the same incident_id for both ALERT and RECOVERY events
  • Example: Monitor "Database Connection" with incident_id: "db-conn-001"
  • First webhook: status: "DOWN" → Creates ALERT record
  • Second webhook: status: "UP" → Creates RECOVERY record
  • Both share the same incident_id but different timestamps

System Behavior

  • Upsert Logic: GitPulse uses (incident_id, project_uuid, incident_time) as the unique key
  • MTTR Calculation: Recovery events automatically find the most recent ALERT with the same project
  • Event Tracking: Each webhook call creates a separate record, preserving the complete incident timeline

Best Practices

  • Consistent Naming: Use descriptive, consistent incident IDs (e.g., "monitor-name-001")
  • Reuse for Related Events: Use the same ID for all events related to the same incident
  • Unique per Project: Ensure incident IDs are unique within each project

Field Descriptions

Required Fields

Field Type Description Example
incident_id string Identifier for the incident (can be reused for related events) "grafana-alert-123"
project_uuid string GitPulse project UUID "550e8400-e29b-41d4-a716-446655440000"
status string Incident status from monitoring tool "DOWN", "FIRING", "UP", "RESOLVED", etc.
incident_time string ISO 8601 timestamp "2025-01-15T10:30:00Z"

Optional Fields

Field Type Description Example
monitor_name string Name of the monitoring system/check "Database Health Check"
monitor_id string ID of the monitoring check "db-health-001"
repo_uuid string GitPulse repository UUID "550e8400-e29b-41d4-a716-446655440001"
message string Human-readable incident description "Service unavailable"
local_time string Local timestamp (if different from incident_time) "2025-01-15 11:30:00"
timezone string Timezone for local_time "Europe/Paris"

Status Values

GitPulse automatically maps various monitoring tool statuses to standard ALERT/RECOVERY values. You can send any of the supported statuses from your monitoring system.

Statuses Mapped to ALERT (Incident Detected)

The following statuses are automatically converted to ALERT:

Status Description Common Usage
DOWN Service is down Uptime monitors, health checks
FIRING Alert is firing Grafana, Prometheus
ALERT Alert triggered Generic alerting systems
WARN Warning condition Various monitoring tools
ACTIVATED Alert activated Custom monitoring systems
TRIGGER Alert triggered Custom monitoring systems
OPEN Incident opened Ticketing systems
PROBLEM_OPEN Problem opened Advanced monitoring systems
PROBLEM Problem detected Various monitoring tools
CRITICAL Critical condition Health checks, monitoring tools
CREATE Incident created Custom systems
UPDATE Incident updated Custom systems
ACTIVE Incident active Various monitoring tools
FAILURE Check failed Health checks, tests
TRIGGERED Alert triggered Various monitoring tools

Statuses Mapped to RECOVERY (Incident Resolved)

The following statuses are automatically converted to RECOVERY:

Status Description Common Usage
UP Service is up Uptime monitors, health checks
RESOLVED Incident resolved Ticketing systems
RECOVERY Service recovered Various monitoring tools
CLOSED Incident closed Ticketing systems
RESOLVE Incident resolved Custom systems
OK Check passed Health checks, monitoring tools
PROBLEM_CLOSE Problem closed Advanced monitoring systems
CLOSE Incident closed Custom systems
SUCCESS Check successful Health checks, tests

Unknown Statuses

If you send a status that's not in the mapping tables, GitPulse will:

  • Keep the original status as-is
  • Log the unknown status for monitoring purposes
  • Process the incident normally

This allows you to use custom statuses while maintaining compatibility with the standard mapping.

Integration Examples

Grafana Alerting

Grafana can send incident notifications using webhook notifications. GitPulse will automatically map Grafana's firing and resolved statuses to ALERT and RECOVERY.

Configure your alert rule with the following webhook settings:

{
  "incident_id": "{{ (index .Alerts 0).Fingerprint }}",
  "project_uuid": "<Your GitPulse Project UUID>",
  "status": "{{ .Status }}",
  "incident_time": "{{ (index .Alerts 0).StartsAt }}",
  "monitor_name": "{{ (index .Alerts 0).Labels.alertname }}",
  "monitor_id": "{{ (index .Alerts 0).Labels.monitor_id }}",
  "repo_uuid": "<Your GitPulse repo UUID>",
  "message": "{{ (index .Alerts 0).Annotations.message }}"
}

Status Mapping: - firingALERT (incident detected) - resolvedRECOVERY (incident resolved)

Grafana Configuration Steps

  1. Create an Alert Rule in Grafana
  2. Add a Webhook notification to your notification policy
  3. Set the webhook URL to: https://your-gitpulse-instance.com/api/incidents/webhook/
  4. Add custom headers:
  5. Content-Type: application/json
  6. X-API-Key: <your-gitpulse-api-key>
  7. Use the template above in the webhook body

Uptime Kuma

Uptime Kuma can send incident notifications for status changes. GitPulse will automatically map Uptime Kuma's status values to ALERT and RECOVERY.

Configure your webhook with:

{  
    "incident_id": "{{ monitorJSON.name }}",  
    "project_uuid": "<Your GitPulse Project UUID>",  
    "status": "{% if heartbeatJSON %}{% if heartbeatJSON.status == 0 %}down{% elsif heartbeatJSON.status == 1 %}up{% elsif heartbeatJSON.status == 2 %}pending{% else %}maintenance{% endif %}{% else %}pending{% endif %}",  
    "incident_time": "{% if heartbeatJSON %}{{ heartbeatJSON.time }}{% else %}{{ 'now' | date: '%Y-%m-%d %H:%M:%S' }}{% endif %}",  
    "local_time": "{% if heartbeatJSON %}{{ heartbeatJSON.localDateTime }}{% else %}{{ 'now' | date: '%Y-%m-%d %H:%M:%S' }}{% endif %}",  
    "timezone": "{% if heartbeatJSON %}{{ heartbeatJSON.timezone }}{% else %}UTC{% endif %}",  
    "monitor_name": "{{ monitorJSON.name }}",  
    "monitor_id": "{{ monitorJSON.id }}",  
    "repo_uuid": "<Your GitPulse repo UUID>",  
    "message": "{{ msg }}"  
}

Status Mapping: - downALERT (service is down) - upRECOVERY (service is up) - pendingALERT (service status unknown) - maintenanceRECOVERY (service in maintenance mode)

Uptime Kuma Configuration Steps

  1. Go to Settings → Notifications in Uptime Kuma
  2. Add a new notification with type "Webhook"
  3. Set the webhook URL to: https://your-gitpulse-instance.com/api/incidents/webhook/
  4. Add custom headers:
  5. Content-Type: application/json
  6. X-API-Key: <your-gitpulse-api-key>
  7. Use the template above in the webhook body
  8. Test the notification to ensure it works

Custom Monitoring Systems

For other monitoring systems, adapt the JSON structure to match your data format:

{
  "incident_id": "your-unique-id",
  "project_uuid": "your-project-uuid",
  "status": "ALERT",
  "incident_time": "2025-01-15T10:30:00Z",
  "monitor_name": "Your Monitor Name",
  "monitor_id": "your-monitor-id",
  "repo_uuid": "your-repo-uuid",
  "message": "Your incident description"
}

Response Format

Success Response

{
  "success": true,
  "message": "Incident recorded successfully",
  "incident_id": "grafana-alert-123"
}

Error Response

{
  "success": false,
  "error": "Invalid project UUID",
  "details": "Project with UUID 'invalid-uuid' not found"
}

Error Handling

Common Error Codes

HTTP Status Error Description Solution
400 Bad Request Invalid JSON or missing required fields Check request format
401 Unauthorized Invalid or missing API key Verify API key
404 Not Found Project or repository not found Check UUIDs
500 Internal Server Error Server processing error Contact support

Validation Rules

  1. incident_id: Can be reused for the same project (e.g., for DOWN/UP events)
  2. project_uuid: Must exist in GitPulse
  3. status: Can be any monitoring tool status (automatically mapped to ALERT/RECOVERY)
  4. incident_time: Must be valid ISO 8601 format
  5. repo_uuid: Must exist in the specified project

Testing the Webhook

Using curl

curl -X POST https://your-gitpulse-instance.com/api/incidents/webhook/ \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-api-key" \
  -d '{
    "incident_id": "test-incident-001",
    "project_uuid": "your-project-uuid",
    "status": "ALERT",
    "incident_time": "2025-01-15T10:30:00Z",
    "monitor_name": "Test Monitor",
    "message": "Test incident for webhook validation"
  }'

Using Postman

  1. Set method to POST
  2. Set URL to your webhook endpoint
  3. Add headers:
  4. Content-Type: application/json
  5. X-API-Key: your-api-key
  6. Set body to raw JSON with your incident data
  7. Send request and verify response

Best Practices

Incident ID Generation

  • Use unique identifiers to avoid duplicates
  • Include timestamp or sequence numbers for uniqueness
  • Use descriptive prefixes for different monitoring systems

Timestamp Handling

  • Always use UTC timestamps for incident_time
  • Use ISO 8601 format for compatibility
  • Consider timezone differences when using local_time

Error Handling

  • Implement retry logic for failed webhook calls
  • Log webhook failures for debugging
  • Use exponential backoff for repeated failures

Security

  • Rotate API keys regularly
  • Use HTTPS for all webhook communications
  • Validate webhook payloads before processing
  • Monitor webhook usage for unusual patterns

Troubleshooting

Webhook Not Working

  1. Check API key in request headers
  2. Verify endpoint URL is correct
  3. Check network connectivity between systems
  4. Review server logs for error messages

Incidents Not Appearing

  1. Verify webhook response indicates success
  2. Check incident data in GitPulse dashboard
  3. Validate JSON format matches expected schema
  4. Confirm project UUID is correct

Performance Issues

  1. Monitor webhook response times
  2. Check server resource usage
  3. Implement rate limiting if needed
  4. Use async processing for high-volume scenarios

Support

If you encounter issues with the incident webhook:

  1. Check the logs for error messages
  2. Verify your configuration matches the examples
  3. Test with curl to isolate the issue
  4. Contact support with detailed error information