Apache skywalking

Overview

This guide details how to integrate Apache SkyWalking with Callgoose SQIBS for real-time incident management, auto-remediation, and event-driven automation. By routing SkyWalking alarm webhooks through your alert pipeline, you can automatically create, update, and resolve incidents based on performance degradation or application anomalies.

Prerequisites

• Apache SkyWalking Account / Instance: Access to configure alarm rules and webhook notifications.
• Callgoose SQIBS Account: Follow the guide on How to Create API Token in Callgoose SQIBS.
• Webhook Endpoint: Refer to the Callgoose SQIBS API Endpoint Documentation to get the endpoint details where the JSON payloads from Falco will be sent.

1. Obtain API Token and Endpoint Details

• Generate an API Token: Create a dedicated API token within your workspace settings.
• Find the API Endpoint: Locate your unique ingestion webhook URL (e.g., [https://your-domain.callgoose.com/v1/process?from=FromValue&token=xxxx](https://your-domain.callgoose.com/v1/process?from=FromValue&token=xxxx)) which will receive the JSON payloads from SkyWalking.

2. Configuring SkyWalking Alarm Webhooks

Apache SkyWalking handles threshold breaches and service metrics through its alarm mechanism. You need to configure the alarm settings to forward notifications via webhook.

• Open Alarm Settings: Locate your SkyWalking backend configuration file, typically named alarm-settings.yml (found within the config directory or mounted via ConfigMap if running on Kubernetes).
• Configure Webhook Rules: Add or update the webhook notification block to point directly to your Callgoose SQIBS ingestion endpoint.
• Example Webhook Configuration block in alarm-settings.yml:
• YAML

webhooks:
  - url: 'https://your-domain.callgoose.com/v1/process?from=SkyWalking&token=xxxx'

3. Example JSON Payloads from SkyWalking

SkyWalking sends structured JSON payloads when an alarm triggers or resolves.

3.1. Alarm Triggered Payload Example

JSON

{
  "scopeId": 1,
  "scope": "SERVICE",
  "name": "service-provider",
  "id": "b3JnLmFwYWNoZS5za3l3YWxraW5n...",
  "ruleName": "service_resp_time_rule",
  "alarmMessage": "Response time of service service-provider is more than 1000ms",
  "startTime": 1718701200000,
  "tags": [
    {
      "key": "layer",
      "value": "GENERAL"
    }
  ]
}

3.2. Alarm Resolved Payload Example

(Note: Depending on your webhook configuration or downstream alert manager, recovery payloads may send status updates or clear parameters matching the active alarm fingerprint).

JSON

{
  "scopeId": 1,
  "scope": "SERVICE",
  "name": "service-provider",
  "id": "b3JnLmFwYWNoZS5za3l3YWxraW5n...",
  "ruleName": "service_resp_time_rule",
  "alarmMessage": "Response time of service service-provider is back to normal",
  "startTime": 1718701200000,
  "tags": [
    {
      "key": "layer",
      "value": "GENERAL"
    }
  ]
}

4. Configuring API Filters in Callgoose SQIBS

To correctly map and manage incidents from incoming SkyWalking webhooks, configure your API filters:

4.1 Integration Templates

• Check the "Select Integration Template" dropdown in your API filter settings. If a SkyWalking template is available, select it to automatically prepopulate the trigger and resolve filter structures. You can then adjust the parameters to fit your workflow requirements.

4.2 Manually Adding and Editing Filters

• Trigger Filter (For Creating Incidents):
• Payload JSON Key: ruleName (or alarmMessage)
• Key Value Contains: service_resp_time_rule (or your specific SkyWalking rule name)
• Map Incident With: id (or name)
• Incident Title From: ruleName
• Incident Description From: Select the alarmMessage key to ensure detailed performance error messages are captured directly in the incident body.
• Resolve Filter (For Resolving Incidents):
• Payload JSON Key: alarmMessage (configured to detect clearance or resolution status text)
• Key Value Contains: back to normal (or your matching resolution keyword)
• Incident Mapped With: id

5. Verifying the Integration

• Trigger a Test Condition: Simulate a high response time or error rate on a service monitored by SkyWalking to force an alarm rule evaluation.
• Check the Platform Dashboard: Log in to your Callgoose SQIBS dashboard to confirm that a new incident has been created automatically with the correct metadata and description.

6. Debugging and Troubleshooting

• Enable API Debugging: Turn on the debugging option when managing your integration API tokens. When enabled, inbound JSON payloads are stored in the platform's API log section for 48 hours.
• Review API Logs: Use the stored request entries in the API log to inspect incoming JSON keys, verify payload formatting, and refine your custom trigger or resolve filters.