Percona

Overview

This document provides a complete guide to integrating Percona Monitoring and Management (PMM) with Callgoose SQIBS using outgoing webhooks.

Percona PMM sends infrastructure and database alert events (CPU, Memory, MySQL down, etc.) to Callgoose SQIBS, where they automatically create incidents through API Filters. This integration uses PMM’s "Contact Points" and Callgoose SQIBS’s Process API endpoint.

Prerequisites

Ensure you have the following before beginning:

• Percona PMM: Active PMM Server (Version 2.31 or higher required for Percona Alerting).
• Callgoose SQIBS:
• API Token.
• API endpoint.
• Configuration: At least one Alert Rule configured in PMM.

1. Obtain Callgoose API Token & Endpoint

• Generate a new token or use an existing one.
• Construct your Callgoose inbound process URL.

URL Format:

https://****.callgoose.com/v1/process?from=PerconaPMM&token=<API_TOKEN>

Note: Replace <API_TOKEN> with your actual token. Store this securely.

2. Webhook Payload Strategy

Unlike some tools that allow custom JSON templates, Percona PMM sends a Fixed JSON Payload structure based on the standard Prometheus Alertmanager format. You cannot modify the top-level structure (e.g., receiver, status, alerts) sent by PMM.

Therefore, the integration strategy relies entirely on Callgoose SQIBS API Filters to map these static fields into the incident structure.

Key Percona PMM Fields:

• status: The overall status of the alert group (e.g., "firing" or "resolved").
• alerts: An array containing details of specific alerts.
• labels: Key-value pairs defining the alert (e.g., alertname, severity, instance).
• annotations: Descriptive text (e.g., summary, description).

3. Configure Percona PMM → Outgoing Webhook

You can configure webhooks via the Alerting interface in PMM.

3.1 Configure Webhook

• Login to your Percona PMM dashboard.
• Navigate to Alerting (Bell Icon) -> Contact points.
• Click + New contact point.
• Name: Enter Callgoose SQIBS.
• Type: Select Webhook from the list.
• Url: Enter the URL constructed in Section 1.
• HTTP Method: Ensure POST is selected.
• Click Save contact point.

Note: After saving, navigate to Notification policies, edit the "Root policy" (or specific nested policies), and assign Callgoose SQIBS as the contact point.

3.2 Add Headers (Not Applicable)

Percona PMM's standard webhook configuration does not require custom HTTP headers for this integration. The authentication is handled by the token passed as a query parameter in the URL as shown in step 3.1.

3.3 Payload Structure

Percona PMM will automatically send a JSON body similar to the structure below. You do not need to configure a template in PMM.

4. Sample Trigger Payload

Percona PMM is a state-based monitoring system. It triggers an alert when a threshold is breached ("firing") and sends an update when it returns to normal ("resolved").

4.1 Trigger (Alert Firing)

{
  "receiver": "Callgoose SQIBS",
  "status": "firing",
  "alerts": [
    {
      "status": "firing",
      "labels": {
        "alertname": "MySQL Down",
        "severity": "critical",
        "instance": "mysql-01:3306"
      },
      "annotations": {
        "summary": "MySQL instance is down",
        "description": "Instance mysql-01:3306 has been unreachable for more than 1 minute."
      },
      "startsAt": "2025-11-26T09:12:00Z",
      "endsAt": "0001-01-01T00:00:00Z"
    }
  ],
  "groupLabels": {
    "alertname": "MySQL Down"
  },
  "commonLabels": {
    "alertname": "MySQL Down",
    "severity": "critical"
  },
  "commonAnnotations": {
    "summary": "MySQL instance is down"
  },
  "externalURL": "http://pmm-server/alerting/list"
}

4.2 Recovery (Alert Resolved)

{
  "receiver": "Callgoose SQIBS",
  "status": "resolved",
  "alerts": [
    {
      "status": "resolved",
      "labels": {
        "alertname": "MySQL Down",
        "severity": "critical"
      },
      "annotations": {
        "summary": "MySQL instance is down"
      },
      "endsAt": "2025-11-26T09:45:00Z"
    }
  ]
}

5. Configure Callgoose SQIBS API Filters

Since Percona PMM provides a fixed payload, you must map the fields in Callgoose.

5.1 Trigger Filter (Create Incident)

Navigate to Integrations -> API Filters -> Create New.

Configure the rules:

• Payload JSON Key: "status"
• Key Value Contains: firing
• Incident Mapped With: "commonLabels"."alertname"
• Title Field: "commonLabels"."alertname - commonAnnotations"."summary"
• Description Field: "alerts".[0]."annotations"."description"

5.2 Resolve Filter (Resolve Incident)

Configuration: This ensures incidents are automatically closed when Percona detects the issue is fixed.

• Payload JSON Key: "status"
• Key Value Contains: resolved (Select "Equals" condition)
• Incident Mapped With: "commonLabels"."alertname"

This ensures:

• Every distinct alert creates a unique incident.
• The incident automatically closes when the database or service recovers.

6. Testing the Integration

• Save the Percona PMM webhook configuration.
• Trigger a Check:
• Navigate to Alerting -> Alert rules.
• Create a temporary "Test" rule (e.g., vector(1) > 0) that evaluates to true immediately.
• Wait for the "For" duration (usually 1 minute) to pass.
• Verify in Callgoose:
• Check API Logs to confirm receipt (HTTP 200 OK).
• Confirm a new incident is created with the title and description populated correctly.
• Cleanup: Delete or mute the temporary alert rule.

7. Security Recommendations

• HTTPS: Always use https:// for the Callgoose URL.
• Token Security: Since the token is passed in the URL (Query Parameter), ensure that you do not share screenshots of your Webhook URL settings.
• Scope: If possible, use a dedicated API token for PMM integrations to easily revoke access if compromised.

8. Troubleshooting

• No Incident Created
• Webhook URL: Ensure the token parameter is correctly appended to the URL (?token=...).
• Filter Mismatch: Check Callgoose API Logs. If the log says "Discarded," ensure your Trigger Filter is looking for the status key with value firing.
• Notification Policy: Ensure your Alert Rule is actually matching a Notification Policy that uses the Callgoose Webhook.
• Missing Description Data
• Field Names: JSON paths are case-sensitive. Ensure you use {{alerts.0.annotations.description}}, not {{Description}}.
• Alert Grouping: If PMM groups multiple alerts into one payload, alerts.0 will only pull the description of the first alert in the batch.
• Webhook Not Received
• Firewall: Ensure no corporate firewall is blocking outgoing requests from the PMM Server container/VM.
• Endpoint: Verify the Callgoose endpoint returns a 2xx status code when tested via curl or Postman from the PMM server CLI.

9. Conclusion

Integrating Percona PMM with Callgoose SQIBS provides a seamless, real-time awareness of database health. By combining PMM’s deep metrics and thresholding with Callgoose’s incident routing, you can ensure that database outages, slow queries, or saturation events are immediately brought to your team's attention.

By using the simple API Filter mappings defined above, this integration ensures that alert events are accurately reflected within Callgoose.

For further customization or advanced use cases, refer to the official documentation for both Percona and Callgoose SQIBS:

• Percona PMM Alerting Documentation
• Callgoose SQIBS API Token Documentation
• Callgoose SQIBS API Endpoint Documentation
• API Filter Instructions and FAQ
• How to Send API