Helicone

Overview

This document provides a detailed guide to integrating Helicone with Callgoose SQIBS for real-time Incident Management, LLM observability–based alerting, and automated incident resolution.

The integration uses Helicone Webhooks to send alert event payloads (latency spikes, error rates, cost anomalies, request failures, SLA breaches, etc.) directly to Callgoose, where API Filters convert them into incidents that are automatically created and resolved.

Prerequisites

Before beginning, ensure you have the following:

• A Callgoose SQIBS account with permissions to create API Filters and access the integration endpoint
• A Helicone account with access to Alerts and Webhooks
• A valid Callgoose API token and Api endpoint URL.
• A test Helicone alert (recommended) for validating webhook delivery and payloads

1. Prepare Callgoose: Obtain Endpoint and Token

1.1 Retrieve Callgoose API Endpoint

Generate or locate your Callgoose process endpoint:

https://****.callgoose.com/v1/process?from=Helicone&token=xxxx

Store the API token securely and never commit it to source control.

2. Configure Helicone Webhook

2.1 Choose the Webhook Scope

You can configure webhooks in Helicone at:

• Organization-level Webhooks
• Settings → Webhooks → Add Webhook
• Alert-level Webhooks
• Alerts → Select Alert → Notifications → Webhook

2.2 Configure Webhook Fields

Set the following options:

• Webhook URL:Callgoose endpoint from Step 1
• HTTP Method:POST
• Content Type:application/json
• Send Resolved Alerts:Must be Enabled for auto-resolution in Callgoose

Save and enable the webhook.

2.3 Verification Headers

Helicone may include verification headers such as:

• X-Helicone-Signature — Signature for payload verification
• X-Helicone-Event — Alert event type
• X-Helicone-Organization — Helicone org identifier

If using an intermediary receiver, you may validate these headers before forwarding events to Callgoose.

3. Example Helicone Payloads

Firing Alert Example

{
  "status": "firing",
  "alert_name": "High LLM Error Rate",
  "alert_id": "helicone-alert-98231",
  "severity": "critical",
  "description": "LLM error rate exceeded 5% threshold",
  "fired_at": "2025-10-01T10:42:00Z",
  "labels": {
    "model": "gpt-4.1",
    "environment": "production"
  },
  "values": {
    "error_rate": 8.4
  }
}

Resolved Alert Example

{
  "status": "resolved",
  "alert_name": "High LLM Error Rate",
  "alert_id": "helicone-alert-98231",
  "resolved_at": "2025-10-01T10:55:18Z"
}

Inspect your actual payloads inside Callgoose API Logs and use those fields in your API Filter mappings.

4. Create API Filters in Callgoose SQIBS

Callgoose API Filters convert incoming Helicone webhooks into incidents.

Create two filters:

• Trigger Filter (create incident)
• Resolve Filter (auto-resolve incident)

4.1 Trigger Filter — Create Incident

Use the following settings:

• Payload JSON Key: status
• Key Value Contains: firing
• Map Incident With: alert_id
• Incident Title: alert_name
• Incident Description: description
• (Optional — leave empty to attach full JSON payload)

This filter creates an incident whenever Helicone fires an alert.

4.2 Resolve Filter — Auto-Resolve Incident

Use the following settings:

• Payload JSON Key: status
• Key Value Contains: resolved
• Incident Mapped With: alert_id

This ensures the incident is resolved when the corresponding Helicone alert clears.

Refer to the API Filter Instructions and FAQ for more details.

5. Verify and Test the Integration

5.1 Test Initial Delivery

• Create a broad test filter in Callgoose
• Trigger a test alert in Helicone
• Confirm Callgoose receives the payload in API Logs

5.2 View Exact Payload

Use Callgoose API Request Logs to inspect the exact JSON payload.

Copy field names directly from these logs for accurate filter configuration.

5.3 Final Verification

• Trigger an alert in Helicone → Incident should be created
• Resolve the alert condition → Incident should auto-resolve
• Confirm mapping consistency using alert_id

6. Troubleshooting

Incoming Payload Not Received

• Check Helicone webhook delivery logs
• Verify Callgoose endpoint URL and token
• Ensure no API Filter mismatch
• Confirm HTTPS connectivity

Incidents Not Auto-Resolving

• Confirm Helicone sends status=resolved
• Ensure the same alert_id is used for firing & resolution
• Verify resolved alerts are enabled in Helicone webhook settings

Wrong or Missing Payload Fields

• Inspect live payloads in Helicone webhook logs
• Update API Filters to match exact field names
• Re-test with firing and resolved alerts

7. Conclusion

Integrating Helicone with Callgoose SQIBS enables automated incident creation and resolution driven by LLM observability signals. By mapping Helicone alert events through Callgoose API Filters, teams gain immediate operational visibility into AI reliability, performance degradation, and cost anomalies.

For further customization or advanced use cases, refer to the official documentation for both helicone and Callgoose SQIBS:

• Callgoose SQIBS API Token Documentation
• Callgoose SQIBS API Endpoint Documentation
• API Filter Instructions and FAQ
• How to Send API
• helicone documentation