CircleCI

Overview

This document provides a detailed guide to integrating CircleCI with Callgoose SQIBS for real-time Incident Management, Incident Auto Remediation, Event-Driven Automation, and other automation purposes. The integration enables automatic creation, updating, and resolution of incidents in Callgoose SQIBS based on build and deployment notifications triggered in CircleCI. The guide includes steps for setting up notifications in CircleCI, configuring webhook notifications, creating API filters in Callgoose SQIBS, and troubleshooting.

Prerequisites

• CircleCI Account: Access to CircleCI for managing build and deployment pipelines.
• Callgoose SQIBS Account: With valid privileges to set up API filters and receive notifications.
• Webhook/API Endpoint: Available in Callgoose SQIBS to receive notifications from CircleCI.

1. Obtain API Token and Endpoint Details

To integrate with Callgoose SQIBS, you first need to obtain an API token and find the API endpoint details.

• Generate an API Token:
• Follow the guide on How to Create API Token in Callgoose SQIBS.
• Find the API Endpoint:
• Refer to the Callgoose SQIBS API Endpoint Documentation to get the endpoint details where the JSON payloads from CircleCI will be sent.

2. Debugging and Troubleshooting

You can enable debugging in the API tokens used with CircleCI notifications for troubleshooting purposes.

• Enable Debugging:
• You can update the debug value when adding or updating an API token.
• When API tracking is enabled, logs are stored in the API log section for your review. The debugging option will automatically disable after 48 hours.
• When API tracking is turned off, no logs are saved in the API log.
• Using API Log for Troubleshooting:
• The API log provides detailed information on all API calls made to Callgoose SQIBS.
• You can check the JSON values in each API log entry for troubleshooting purposes.
• Use the information in the API log to create or refine API filters to ensure incidents are created correctly based on the API payloads received.
• Callgoose SQIBS creates incidents according to your API filter configuration, giving you full control over how alerts from different services trigger incidents and alerts for your support team or automation processes.

3. Configuring CircleCI to Send JSON Payloads

These steps will guide you through setting up the necessary notifications within CircleCI to ensure that the JSON payloads match those expected by Callgoose SQIBS.

3.1 Setting Up CircleCI

To generate the required JSON payloads, you first need to set up CircleCI.

• Log in to the CircleCI Console:
• Access the CircleCI platform using your account credentials.
• The authorization method used to set up your CircleCI account determines the definition of project in CircleCI.
• On the Organization Home page, check which option you see:
• If you see a Set up a project button, follow steps to Set up a project.
• If you see a Create Project button, follow steps to Create a project.
• Navigate to the Project Settings:
• In the CircleCI projects section, go to the Project you want to configure webhooks for.
• Click on Project Settings.
• Click on Webhooks.

3.2 Configuring the Webhook Notification

• Add Webhook URL:
• Click on Add Webhook.
• Provide the Webhook name and Receiver URL.
• Receiver URL is the API Endpoint URL provided by Callgoose SQIBS. Refer to the API Endpoint documentation for more details on selecting final endpoint format.
• Select the relevant events, such as "Workflow Completed."
• Add the webhook.
• Test the Webhook Configuration:
• Perform a test to ensure that the JSON payload is correctly formatted and is being sent to the Callgoose SQIBS API endpoint as expected.
• After adding the webhook click on the webhook you created.
• Click on Test Ping Event to send a test payload to Callgoose SQIBS.
• Review the payload in Callgoose SQIBS to confirm that it matches the expected structure.

3.3 Finalizing and Testing

• Validate the Integration:
• Trigger the condition manually if possible to verify that the correct JSON payload is sent to Callgoose SQIBS.
• Resolve the trigger to ensure the resolved state payload is also correctly sent and processed.

4. Configuring Callgoose SQIBS

4.1 Create API Filters in Callgoose SQIBS

To correctly map incidents from the CircleCI notifications, you need to create API filters based on the JSON payloads received.

4.1.1 Example JSON Payloads from CircleCI

Workflow Completed Triggered (status: "failed"):

json

{
  "type": "workflow-completed",
  "id": "d3ab****-0444-3f00-9ec3-5712b4****",
  "happened_at": "2024-09-11T08:54:35.536486Z",
  "webhook": {
    "id": "0ca****-942a-499d-af1b-1c0217a****",
    "name": "Testing"
  },
  "workflow": {
    "id": "****626d-4f0e-4cd5-a5ee-50600a69****",
    "name": "say-hello-workflow",
    "created_at": "2024-09-11T08:54:30.063Z",
    "stopped_at": "2024-09-11T08:54:35.394Z",
    "url": "https://app.circleci.com/pipelines/bitbucket/****/test-2/23/workflows/****-4f0e-4cd5-a5ee-****",
    "status": "failed"
  },
  "pipeline": {
    "id": "****-276a-4d16-b18e-****",
    "number": 23,
    "created_at": "2024-09-11T08:54:29.891Z",
    "trigger": {
      "type": "webhook"
    },
    "vcs": {
      "provider_name": "bitbucket",
      "origin_repository_url": "https://bitbucket.org/lloyd0605/test-2",
      "target_repository_url": "https://bitbucket.org/lloyd0605/test-2",
      "revision": "****be738de1581b6b83e****",
      "commit": {
        "subject": "config.yml edited online with Bitbucket",
        "body": "",
        "author": {
          "name": "****",
          "email": "****@gmail.com"
        },
        "authored_at": "2024-09-11T08:54:27Z",
        "committer": {
          "name": null,
          "email": null
        },
        "committed_at": null
      },
      "branch": "circleci-project-setup"
    }
  },
  "project": {
    "id": "****-afef-4c81-b3d0-****",
    "name": "test-2",
    "slug": "bitbucket/****/test-2"
  },
  "organization": {
    "id": "****-0f13-4c16-94b0-****",
    "name": "****"
  }
}

Workflow Completed Triggered (status: "success"):

json
	
{
  "type": "workflow-completed",
  "id": "****-e754-3c7e-9ced-****",
  "happened_at": "2024-09-11T08:57:52.958890Z",
  "webhook": {
    "id": "****-942a-499d-af1b-****",
    "name": "Testing"
  },
  "workflow": {
    "id": "****-ba85-4150-a336-****",
    "name": "say-hello-workflow",
    "created_at": "2024-09-11T08:57:24.847Z",
    "stopped_at": "2024-09-11T08:57:52.831Z",
    "url": "https://app.circleci.com/pipelines/bitbucket/lloyd0605/test-2/24/workflows/0aaa2aaf-ba85-4150-a336-c5e200107414",
    "status": "success"
  },
  "pipeline": {
    "id": "****-6e2a-4c23-915f-****",
    "number": 24,
    "created_at": "2024-09-11T08:57:24.647Z",
    "trigger": {
      "type": "webhook"
    },
    "vcs": {
      "provider_name": "bitbucket",
      "origin_repository_url": "https://bitbucket.org/lloyd0605/test-2",
      "target_repository_url": "https://bitbucket.org/lloyd0605/test-2",
      "revision": "42b1231a1fe38deabd4c7687dc4bc****",
      "commit": {
        "subject": "config.yml edited online with Bitbucket",
        "body": "",
        "author": {
          "name": "****",
          "email": "****@gmail.com"
        },
        "authored_at": "2024-09-11T08:57:21Z",
        "committer": {
          "name": null,
          "email": null
        },
        "committed_at": null
      },
      "branch": "circleci-project-setup"
    }
  },
  "project": {
    "id": "****-afef-4c81-b3d0-****",
    "name": "test-2",
    "slug": "bitbucket/****/test-2"
  },
  "organization": {
    "id": "****-0f13-4c16-94b0-****",
    "name": "****"
  }
}

4.2 Configuring API Filters

4.2.1 Integration Templates

If you see a CircleCI integration template in the "Select Integration Template" dropdown in the API filter settings, you can use it to automatically add the necessary Trigger and Resolve filters along with other values. The values added by the template can be modified to customize the integration according to your requirements.

4.2.2 Manually Add/Edit the Filter

• Trigger Filter (For Creating Incidents):
• Payload JSON Key: "workflow"."status"
• Key Value Contains: [failed]
• Map Incident With: "project"."id"
• This corresponds to the project id from the CircleCI payload.
• Incident Title From: "pipeline"."vcs"."commit"."subject"
• This will use the commit subject as the incident title in Callgoose SQIBS.
• You can also configure the API endpoint URL from Callgoose Sqibs using the 'from' URL parameter. The 'from' parameter value will be used if the payload does not contain the specific Incident title key specified in the API filter. Refer to the API endpoint documentation for more details.
• Incident Description From: Leave this empty unless you want to use a specific key-value from the JSON payload. If a key is entered, only the value for that key will be used as the Incident Description instead of the full JSON. By default, the Incident Description will include the full JSON values.
• Example: If you use the "workflow" key in the Incident Description From field, the incident message will be the value of the "workflow" key. In the example JSON payload provided earlier, this would result in a description like:

"workflow": {
"id": "****-ba85-4150-a336-****",
"name": "say-hello-workflow",
"created_at": "2024-09-11T08:57:24.847Z",
"stopped_at": "2024-09-11T08:57:52.831Z",
"url": "https://app.circleci.com/pipelines/bitbucket/lloyd0605/test-2/24/workflows/0aaa2aaf-ba85-4150-a336-c5e200107414",
"status": "success"
},

• Resolve Filter (For Resolving Incidents):
• Payload JSON Key: "workflow"."status"
• Key Value Contains: [success]
• Incident Mapped With: "project"."id"
• This ensures the incident tied to the specific project id is resolved when the build or deployment status is successful.

4.3 Finalizing Setup

• Save the API Filters:
• Ensure that the filters are correctly configured and saved in Callgoose SQIBS.
• Double-check that all key mappings, incident titles, and descriptions are correctly aligned with the payload structure sent by CircleCI.

Refer to the API Filter Instructions and FAQ for more details.

5. Testing and Validation

5.1 Triggering Builds and Deployments

• Simulate a Build/Deployment:
• Trigger a build or deployment in CircleCI to simulate a development or operational task.
• Verify that an incident is created in Callgoose SQIBS with the correct information.

5.2 Resolving Incidents

• Complete the Build/Deployment:
• Once the build or deployment is completed in CircleCI, verify that the incident in Callgoose SQIBS is updated or marked as resolved based on the success or failure status of the build.
• Check Incident Resolution:
• Ensure that the incident created in Callgoose SQIBS is resolved when the build or deployment completes successfully in CircleCI.
• If the build or deployment fails again, a new incident should be created or the existing one updated depending on your API filter configuration.

6. Security Considerations

• API Security: Ensure that the Callgoose SQIBS API endpoint is correctly configured and that the API token is securely stored and used.
• CircleCI Permissions: Confirm that the webhook in CircleCI has appropriate permissions to send notifications and data to Callgoose SQIBS.

7. Troubleshooting

• No Incident Created: If no incident is created, verify that the webhook URL in CircleCI is correct and that the JSON payload structure matches the API filters configured in Callgoose SQIBS.
• Incident Not Resolved: Ensure that the resolve filter in Callgoose SQIBS is correctly configured and that the JSON payload sent by CircleCI matches the expected structure.

8. Conclusion

This guide provides a comprehensive overview of how to integrate CircleCI with Callgoose SQIBS for effective incident management. By following the steps outlined, you can ensure that build and deployment notifications from CircleCI are automatically reflected as incidents in Callgoose SQIBS, with proper resolution tracking when the issues are resolved.

For further customization or advanced use cases, refer to the official documentation for both CircleCI and Callgoose SQIBS:

• CircleCI Documentation
• Callgoose SQIBS API Token Documentation
• Callgoose SQIBS API Endpoint Documentation
• API Filter Instructions and FAQ
• How to Send API

This documentation will guide you through the integration process, ensuring that your incidents are managed effectively within Callgoose SQIBS based on real-time notifications from CircleCI.