Mapping Integration Data to Custom Properties

Custom Integrations each come equipped with a webhook endpoint that allow you to push data to OpsLevel. Users can then manipulate this data via Extraction and Transform Configurations to update Custom Properties on existing Component Types.

Rather than relying on pre-defined integrations, this capability enables you to fully customize how data from external tools is incorporated into your catalog.

Why is Customizable Data Mapping Important?

Flexibility: Map data from various sources (like Wiz, Jira, or Datadog) to OpsLevel in the way that makes the most sense for your organization.
Faster Integration: Bring in data from tools that OpsLevel doesn't natively support, without waiting for bespoke development.
Improved Data Visibility: Control where your data is stored and how it's displayed, giving you a clearer picture of your components and systems.

Getting Started

You will need to set up a Custom Integration in order to get started. This can be done through the UI or the API by following the guide found here. The Customizable Data Mapping feature configuration can be found within your Custom Integration, in the Extract and Transform Configuration collapsible section.

Within the integration details card, there is a Webhook URL. This is the endpoint you'll use to push your data to OpsLevel.

The expected structure of the webhook query parameters:

Keys	Required	Description
`external_kind`	true	A unique identifier string, used by the Customizable Data Mapping feature to identify the type of your data.

Configuration

The steps below follow a scenario where we want to push a Jira Epic to OpsLevel. Once configured, our user will send a POST request to the endpoint https://app.opslevel.com/integrations/custom/webhook/••••••••••••••••••••••••••••••••••?external_kind=jira_epic_push_data, including the data:

{
  "key": "APA-1123",
  "id": "10000000",
  "issues": [
    {
      "fields": {
        "status": {
          "name": "Open"
        }
      }
    },
    {
      "fields": {
        "status": {
          "name": "In Progress"
        }
      }
    }
  ]
}

Extraction Definition

You will have to define an Extraction Definition within the Custom Integration in order for OpsLevel to store any data received via the integration. The Extraction Definition is written in YAML, and specifies how data sent to the Custom Integration will be stored for use in the transformation phase of the integration.

The expected structure of the Extraction Configuration is as follows:

---
extractors:
- external_kind: jira_epic_push_data
  external_id: ".id"

Keys	Required	Description
`external_kind`	true	A unique identifier string, used to select the extractor to apply. This string will be stored with the transformed data. and is used within OpsLevel to identify the data type.
`external_id`	true	A JQ expression used to select the unique identifier that will represent the transformed data within OpsLevel.
`iterator`	false	A JQ expression used to select the field to iterate on. If this key is specified, it will create individual objects within OpsLevel for each item iterated on, and all the other keys will apply per-object. If this key is not specified, the entire data payload will be treated as a single object.
`exclude`	false	A JQ expression used to exclude certain objects from processing.
`expires_after_days`	false	An integer number of days. Objects which have not received an updated payload within the specified number of days will be automatically deleted.

Our user has defined a jira_epic_push_data kind that will be uniquely identified within OpsLevel by the value in theid field as specified.

Transform Definition

Transform Definitions must be created to manipulate OpsLevel objects with pushed data. The Transform Definition is written in YAML, and defines how extracted data is to be mapped to OpsLevel Component Types and Teams.

The expected structure of the Transform Definition is as follows:

Keys	Required	Description
`external_kind`	true	A unique identifier string, used to select the extracted data to transform. This should match an `external_kind` defined in the Extraction Definition.
`opslevel_kind`	true	A string used identify the type of component that this transform will update. This should match the alias of a Component Type or be equal to "team".
`opslevel_identifier`	true	A JQ expression used to select an identifier from the extracted data. The result of this expression should match the alias of an existing Component or Team.
`on_component_not_found`	true	An enum describing the action OpsLevel should take should the component identified by the `opslevel_kind` and `opslevel_identifier`. OpsLevel is able to skip the component, create a new component or create a suggestion for a component using the values `skip`, `create` and `suggest` respectively.
`properties`	true	A key, containing a list of key-value pairs, where each key is the identifier of a custom property on the component type within OpsLevel. The value should be a JQ expression, which will be evaluated against the extracted data.

A sample Transform Definition can be found below.

---
transforms:
- external_kind: jira_epic_push_data
  opslevel_kind: jira_epic
  opslevel_identifier: ".key"
  on_component_not_found: "skip"
  properties:
    open_issues: "[.issues[] | {Status: .fields.status.name}] | group_by(.Status) | map({Status: .[0].Status, open_issues: length}) | .[] | select(.Status == \"Open\") | .open_issues"

Currently, this feature exclusively supports existing Component Types and Teams within OpsLevel. In order for your Transform Definition to take effect, please follow the linked Component Types and Teams setup guides.

Our user has created a Transform Definition that will update ajira_epic Component Type. The open_issues property will be updated with a count of all the issues with status "Open". The transform will be applied to all pushed data of the jira_epic_push_datakind.

Our user has setup the requisite data as seen below. Notice the following:

The alias of the "Jira Epic" matches theopslevel_kind in the Transform Definition
The identifier of the Custom Property is open_issues, which matches the property in the Transform Definition
An existing "Jira Epic" Component has been created with the alias apa-1123, matching the result of the opslevel_identifier JQ expression when evaluated against the JSON shown above.

Putting it all Together

Once your Extraction and Transform Definitions have been saved, you may now push your data to the webhook endpoint shown in the details card.

Our user will be making a POST request to the endpoint https://app.opslevel.com/integrations/custom/webhook/••••••••••••••••••••••••••••••••••?external_kind=jira_epic_push_data.

Once the pushed data has been extracted and transformed, the Custom Property on the specified Component will be updated. A tooltip will appear to indicate that the property is managed by an integration.