Integrations
Big Eye
Overview
This document provides a detailed guide to integrating BigEye with Callgoose SQIBS for real-time Incident Management, automated incident creation, and alert-driven workflows.
BigEye provides comprehensive data observability with monitoring for anomaly detection, freshness, volume, schema changes, SLAs, and custom metrics.
By connecting BigEye alert notifications to Callgoose SQIBS through webhook integrations, alerts generated in BigEye will automatically create, update, and resolve incidents inside Callgoose.
This guide covers configuring BigEye Webhooks, creating Incident Filters in Callgoose SQIBS, testing, and troubleshooting.
Prerequisites
- BigEye account with sufficient permissions to manage Validations, Alerts, Integrations, and Workspace Settings.
- Callgoose SQIBS account with permissions to create API Filters.
- A Callgoose SQIBS Webhook/API endpoint.
- A valid Callgoose API token.
1. Obtain API Token and Endpoint Details
1.1 Generate an API Token
Create an API token inside Callgoose SQIBS. Save it securely for use in the integration.
1.2 Find the API Endpoint
Your Callgoose SQIBS endpoint will follow this format:
https://****.callgoose.com/v1/process?from=BigEye&token=xxxx
Replace the token with the API Token generated previously.
2. Configuring BigEye to Send Alerts
BigEye supports webhook-based alerting through its Notification Destinations.
You will configure a webhook destination that forwards alert payloads to Callgoose SQIBS.
2.1 Install and Set Up BigEye (If Not Configured)
- Log into your BigEye console.
- Ensure you have access to:
- Workspace Settings
- Integrations
- Alert Rules
- Monitors and Validations
2.2 Create a Webhook Notification Destination
BigEye supports adding custom Webhook endpoints that send JSON alert payloads.
Steps:
- Navigate to Settings → Notifications / Integrations.
- Click Add Integration.
- Select Webhook or Custom Webhook.
- Fill in the details:
- Integration Name: Callgoose SQIBS
- URL: Callgoose endpoint from Step 1.2
- Method: POST
- Content Type: application/json
- Headers: (Optional)
- If sending token via header, you may use a custom header such as:
- Authorization: Bearer token
- Save the integration.
2.3 Attach Webhook to BigEye Alerts
BigEye allows attaching integrations to notification policies for individual monitors.
Steps:
- Navigate to the dataset or table you are monitoring.
- Choose a validation or create a new one (e.g., freshness check, distribution anomaly, schema drift).
- Go to Alerting.
- Enable Notification and select the Callgoose SQIBS Webhook integration.
- Save the validation.
2.4 Test the Webhook
BigEye supports sending test messages:
- Navigate to Settings → Integrations.
- Select the Callgoose SQIBS integration.
- Click Send Test Alert.
- Confirm that Callgoose SQIBS receives the test payload.
3. Example JSON Payloads from BigEye
BigEye emits webhook alerts when data quality rules fail or return to normal.
Typical fields include dataset, rule name, severity, and state.
3.1 Triggered Alert Payload
{
"alert_name": "Freshness Failure - Customers Table",
"dataset": "customers",
"column": "updated_at",
"severity": "CRITICAL",
"rule_type": "Freshness",
"value": "3 hours delay",
"threshold": "1 hour",
"timestamp": "2025-01-10T12:30:00Z",
"status": "triggered",
"metric": "record_freshness",
"environment": "production"
}
3.2 Resolved Alert Payload
{
"alert_name": "Freshness Recovered - Customers Table",
"dataset": "customers",
"column": "updated_at",
"severity": "INFO",
"rule_type": "Freshness",
"timestamp": "2025-01-10T12:45:00Z",
"status": "resolved",
"environment": "production"
}
BigEye may include additional metadata depending on the rule type.
4. Configuring Callgoose SQIBS
Callgoose SQIBS API Filters allow mapping of BigEye alert webhook payloads to incidents.
4.1 Create API Filters
4.1.1 Trigger Filter (For Creating Incidents)
- Payload JSON Key: "status"
- Key Value Contains: triggered
- Map Incident With: "alert_name" or "dataset" (recommended: use dataset for correlation)
- Incident Title: "alert_name"
- Incident Description: Leave empty to use full payload
This filter will create a new incident whenever BigEye triggers an alert.
4.1.2 Resolve Filter (For Resolving Incidents)
- Payload JSON Key: "status"
- Key Value Contains: resolved
- Incident Mapped With: same key used in Trigger Filter (e.g., "dataset")
This ensures each incident resolves when BigEye indicates the issue has cleared.
5. Testing and Validation
5.1 Triggering Alerts
To test the integration:
- Lower thresholds temporarily for freshness or volume.
- Trigger an anomaly intentionally.
- Confirm that:
- BigEye sends a webhook to Callgoose SQIBS
- Callgoose creates a new incident
5.2 Resolving Alerts
- Revert thresholds or fix the root condition.
- BigEye will send a resolved event.
- Confirm that Callgoose:
- Receives the resolved payload
- Automatically resolves the incident
6. Debugging and Troubleshooting
- No incident created:
- Ensure webhook URL is correct
- Verify Trigger Filter matches "status": "triggered"
- Incident not resolving:
- Ensure the Resolve Filter checks for "status": "resolved"
- Confirm both Trigger and Resolve filters use same mapping key
- Malformed JSON or Missing Fields:
- Check BigEye payload templates
- Confirm BigEye monitor supports the selected fields
- Authentication errors:
- Validate API token
- Ensure query token or header is applied correctly
7. Security Considerations
- Keep the Callgoose API token private.
- Use only HTTPS endpoints.
- Rotate API tokens regularly.
- Limit BigEye access to necessary teams.
8. Conclusion
By following this guide, you can seamlessly integrate BigEye with Callgoose SQIBS.
BigEye’s data quality alerts will now trigger automatic incident creation and resolution inside Callgoose, enabling faster detection, improved visibility, and streamlined data operations workflows.
This integration ensures that every data anomaly or recovery in BigEye becomes an actionable and trackable incident in Callgoose SQIBS.
For further customization or advanced use cases, refer to the official documentation for both Big Eye and Callgoose SQIBS:
